Restructure recorded_data folder

Maritime-Robotics-Student-Society / sailing-robot

Southampton sailing robot

http://www.sotonsailrobot.org

Other

87 stars 46 forks source link

Restructure recorded_data folder #260

Open Nanoseb opened 5 years ago

Nanoseb commented 5 years ago

For now all the logs are in the same folder, which lead to 1300 files in one place. The filenames not starting with the timestamp but their content (like param-dump) is not helping the sorting. To improve that I thought about having a folder per launch with the name: 2018-08-28T12.41.55-calshot-triangular-race And then in it:

rosbag.bag
param-dump.json
gps-raw-nmea
gps-trace.csv
map-gps-trace.html
rosbag.bag.metadata

When/If we changes this file structure it will lead to adapting the launch files that stores bags etc. as well as the index_recorded_data.py script that generates the html page and .metadata.

takluyver commented 5 years ago

The index was designed to alleviate this a bit, but I agree that it would make sense to arrange the files into folders as well.

Nanoseb commented 5 years ago

Also having this kind of folder structure means that we can have a script that easily in one command: run roscore, load the parameter dump, play the rosbag and launch the 2D visualisation for example. Which is tedious at the moment because each file has a different name structure.

Nanoseb commented 5 years ago

What needs to be edited:

[ ] path/filename in script/debugging_dump_params
[ ] path/filename in script/debugging_gps_log
[ ] path/filename in launchfile.launch (rosbag record line)
[ ] path/filename in script/sensor_driver_gps for raw-nmea file

One of the difficult aspect here is to have a consistent timestamp that is propagated and used by all the files (=how to set a parameter with the timestamp of the beginning of the launch)

And also:

[ ] index_recorded_data.py file handling
[ ] write a one time script to convert the old file structure to the new one (part of index_recorded_data.py can be reused to associate file belonging to the same mission together)

smaria commented 5 years ago

IIRC there is no easy way of making rosbag record append a postfix rather than using a prefix. For Delphin2 I ended up using a bash script for launching, which recorded the start time and then handled all the naming at the end.