LouKordos / walking_controller

The main walking controller code for the Bipedal Robot.
13 stars 2 forks source link

Improve / Refactor Logging solution #5

Open LouKordos opened 3 years ago

LouKordos commented 3 years ago

The current logging solution is a simple CSV dump at every loop iteration. It is not robust at all this way and the MPC logs for example are overwritten on every run of the walking controller.

To fix this, explore C++ InfluxDB interfaces (IMPORTANT: Use InfluxDB 2.0 to make it future proof) to write all data to an InfluxDB server, preferably running as a docker container, combined with a tool like Graphana or the TICK stack for inspecting the plots.

The other approach was discussed in the Cpplang slack and involved a binary log, copying over the selected values directly into the file, with an asynchronous IO library writing a ring buffer to a file. This would allow huge data rates (relatively speaking), making it extremely future proof, but also quite overkill. A downside is the likely need of a separate script / program that parses that binary log and makes it usable in plotting software.

In both cases a central, asynchronous Logging Class / Library should be developed and integrated into the code so that an arbitrary thread is able to write data to the main log, timestamped and tagged to associate it with that thread.

LouKordos commented 3 years ago

Batch-writing the data would be very beneficial in general to reduce write cycles on the drive.

LouKordos commented 3 years ago

For logging to a file, realtime compression using zstd would also be very beneficial as logfile sizes are starting to increase.