Use TUM rgbd dataset evaluate rtabmap_ros

[Wei0703]

Excuse me,

I want to use TUM dataset to evaluate absolute trajectory error (ATE) metric. I use $ rtabmap-report --poses rtabmap.db command to get rtabmap_odom.txt and rtabmap_slam.txt. And use TUM dataset tools to get ATE result, command like this $ python evaluate_ate.py groundtruth.txt rtabmap.txt. Following picture are results, but I think the results are wrong.

Am I doing something wrong? By the way, I use rosbag publish topic to rtabmap_ros node. groundtruth.txt download from TUM rgbd dataset fr3_walking_xyz sequence.

[matlabbe]

Can you run the evaluation tool with "--plot figure.png"? If you want more poses to compare, launch rtabmap node with parameter Rtabmap/CreateIntermediateNodes at true.

If you test under ros, see also this post using rgbdslam_datasets.launch.

[Wei0703]

Yes, following picture is my evaluation result.

I try to use rgbdslam_datasets.launch, but loop closure detection window can't display anything. How do I export more poses to compare. My rtabmapviz screen like this.

Thanks a lot!

[matlabbe]

Which dataset are you testing? I could give a try. It looks like the rgbd_dataset_freiburg3_sitting_xyz.bag dataset. Following those instructions, I got this:

However, looking at the database, we don't have all poses in the map (gray is ground truth):

It is because of RGBD/LinearUpdate parameter that is 10 cm by default (rtabmap won't add new nodes if the camera didn't move more than 10 cm). I updated the launch file to keep everything. Here the new results:

If you want to export all poses from rtabmapviz, make sure to click Pause, then you have to click on Edit->Download Graph and select Global Optimized. You can then do File->Export poses...

Another easy way to get poses and ground truth (after closing launch):

$ rtabmap-report --poses ~/.ros/rtabmap.db 

Database: /home/mathieu/.ros/rtabmap.db
   rtabmap.db (363, s=1.000):   error lin=0.019m (max=0.027m, odom=0.019m) ang=0.7deg, slam: avg=36ms (max=76ms) loops=0, odom: avg=0ms (max=0ms), camera: avg=0ms, map=-1MB

It is known that camera calibration or depth scale is a little off with this dataset, to automatically scale the poses:

$ rtabmap-report --poses --scale ~/.ros/rtabmap.db 

Database: /home/mathieu/.ros/rtabmap.db
   rtabmap.db (363, s=1.007):   error lin=0.018m (max=0.027m, odom=0.018m) ang=0.7deg, slam: avg=36ms (max=76ms) loops=0, odom: avg=0ms (max=0ms), camera: avg=0ms, map=-1MB

rtabmap_gt.txt rtabmap_slam.txt

With evaluation script:

$ python evaluate_ate.py --plot figure.png ~/.ros/rtabmap_gt.txt ~/.ros/rtabmap_slam.txt 
0.018371

[SYLJSTK]

$ rtabmap-report --poses --scale ~/.ros/rtabmap.db 

Database: /home/mathieu/.ros/rtabmap.db
   rtabmap.db (363, s=1.007):   error lin=0.018m (max=0.027m, odom=0.018m) ang=0.7deg, slam: avg=36ms (max=76ms) loops=0, odom: avg=0ms (max=0ms), camera: avg=0ms, map=-1MB

When I run this command, I can't find rtabmap_gt.txt ，How can i get it？

[matlabbe]

The path is defined here: https://github.com/introlab/rtabmap/blob/0f221ba3cd9913b68ec508af0d914e3f49860430/tools/Report/main.cpp#L934

It should be exported under ~/.ros