.png)
Friday, April 4, 2008 at 9:50 AM
Part 1 talked about our datalogging hardware, this time we'll focus on some of the technical details of the software.
The dataloggers contain a 4GB flash card with a minimal Gentoo Linux installation. We've stripped down the startup process so that the boot times are as small as possible. We run an SSH daemon (which allows us to control the datalogger remotely), the datalogging program, and almost nothing else.
The flash is partitioned into a system area and a 128MB log partition. Since the datalogger can lose power at any time, to avoid corruption of the root filesystem, that filesystem is always read-only.
The log is organized into 4K blocks where each block is checksummed and numbered. At boot time the datalogger verifies every block and builds a list of all the uncorrupted blocks on the flash, in order. It also notes the greatest block number found and starts numbering new blocks from that point. Corruption of the flash is detected and silently ignored.
The datalogger code itself is written in C, using the open source libevent library (written by another Googler). It runs an HTTP server for debugging, runs the sensors and writes a log frame once a second. Frames are packed into 4K blocks and blocks are written to the flash once full. Writing and uploading are handled by a separate process to ensure that the main process doesn't get stalled for too long. The datalogger is also the klogd and syslogd for the system. If it crashes, it gets restarted a second later and it records its core files in the log for later analysis.
Uploading happens over WiFi whenever the car returns to Google. A daemon watches for a special wireless network and signals the datalogger to start uploading. Each car uses its SSH host key to authenticate, over HTTP, to a server listening at Google. The Google server knows the last block number that was uploaded and the car uploads as many blocks as it can, starting from that number.
The frames are stored in a Bigtable. Here's what a tiny part of one of the resulting frames looks like:
Frame 1204840857 (State (CarSample {datum = [Datum {datum_type = GPS_OPEN, datum_value = 1.0},Datum {datum_type = SPEED, datum_value = 0.987980195657361},Datum {datum_ type = TARGET_RPM, datum_value = 1504.0},Datum {datum_type = FUEL_FLOW, datum_value = 565.0},Datum {datum_type = BRAKES, datum_value = 15.0},Datum {datum_type = STEERING_ANGLE, datum_value = 220.0},...
All those frames are processed to group them into meaningful events, like journeys and charging periods. Then we put them all on the RechargeIT website. Our most useful information is probably the MPG that the car achieves. But, as we'll talk about in another blog post, those numbers should be interpreted in context...
