Select public datasets to benchmark Beluga

[nahueespinosa]

Description

As part of #163, we need to define which input datasets to use for benchmarking. We should aim for selecting a subset of the available datasets that represent different localization conditions. The nature of those conditions remains to be defined, but in principle:

• Environments could be classified by their symmetry, indoor/outdoor, dynamic/static, etc.
• Sensor data could have different characteristics (i.e. precision, resolution, publish rate?).
• Ground-truth data might be available or not (for our application this is a must-have).

Regarding sensor configuration, we currently support 2D LIDAR + odometry.

The common format will most likely need to be a ROS 2 bag, so some preprocessing scripts will be needed.

Definition of done

• [ ] We have a list of datasets pre-processed and ready to use with LAMBKIN.

Additional considerations

We have collected public datasets in the past (although the pre-processing was done with ROS 1 in mind), see https://github.com/ekumenlabs/RnD/issues/5.

We should also check out new collections, for example: https://github.com/mint-lab/awesome-robotics-datasets.

[nahueespinosa]

Public datasets rarely come with a map, so we will need to create it from the data. LAMBKIN is already capable of running SLAM with the following datasets:

• Magazino: https://google-cartographer-ros.readthedocs.io/en/latest/data.html#magazino
• MARS: https://robotics.shanghaitech.edu.cn/datasets/MARS-Dataset
• Rawseeds: http://www.rawseeds.org/home/category/benchmarking-toolkit/datasets/
• TUM: https://vision.in.tum.de/data/datasets/rgbd-dataset/download

[hidmic]

@nahueespinosa @serraramiro1 FYI rosbags-convert can be used to migrate datasets from ROS 1 bag format to ROS 2 bag format.

[serraramiro1]

Yes! I've tried it, and aside from having issues with latched topics, for instance /tf_static, it works really well.

I had the same output with this python package https://gitlab.com/ternaris/rosbags which is way easier to install as it doesn't require ROS.

[serraramiro1]

I've decided to start with the magazino datasets. Reason being:

They're short, and manageable from a provisioning POV.

git-lfs has a strong limit at 5GB https://docs.github.com/en/enterprise-server@3.5/repositories/working-with-files/managing-large-files/about-git-large-file-storage#about-git-large-file-storage , so I'll go with those and then carefully upload some others.

[hidmic]

I think we can call this done, as generic as the description is.