search

rpng / open_vins

An open source platform for visual-inertial navigation research.
https://docs.openvins.com
GNU General Public License v3.0
2.22k stars 652 forks source link

The metrics evaluation is not working #462

Closed setareh-soltanieh closed 3 days ago

setareh-soltanieh commented 3 months ago

I want to compare the estimated pose with the ground truth pose using ov_eval. The dataset that I am using is the EuRoC dataset. I am recording a rosbag of the estimated pose data (/ov_msckf/poseimu) and the ground truth poses (for the vicon_room1 is: /vicon/firefly_sbx/firefly_sbx). After that I convert this rosbag into two txt files (stamped_traj_estimate.txt and stamped_groundtruth.txt). After that I run the following command to plot my results.

rosrun ov_eval plot_trajectories stamped_groundtruth.txt stamped_traj_estimate.txt Figure_1 Figure_2

This command works and it will plot the ground truth and the aligned estimated paths correctly. but when I want to capture the ATE and RPE errors using the following command:

rosrun ov_eval error_singlerun stamped_groundtruth.txt stamped_traj_estimate.txt

The results don't make sense!

======================================
Absolute Trajectory Error
======================================
rmse_ori = 178.702 | rmse_pos = 0.103
mean_ori = 178.701 | mean_pos = 0.097
min_ori  = 176.714 | min_pos  = 0.016
max_ori  = 179.984 | max_pos  = 0.173
std_ori  = 0.547 | std_pos  = 0.037
======================================
Relative Pose Error
======================================
seg 8 - median_ori = 143.450 | median_pos = 4.591 (2383 samples)
seg 16 - median_ori = 88.770 | median_pos = 3.702 (2082 samples)
seg 24 - median_ori = 92.984 | median_pos = 3.481 (1816 samples)
seg 32 - median_ori = 85.973 | median_pos = 3.481 (1427 samples)
seg 40 - median_ori = 138.841 | median_pos = 3.082 (1099 samples)

The orientations are almost 180 degrees incorrect. Can someone please tell me what is the issue?

goldbattle commented 3 months ago

Hi, you shouldn't directly use the /vicon/firefly_sbx/firefly_sbx topic since this contains the transform from the mocap world frame to the mocap marker frame. This isn't the IMU sensor frame we estimate in. You can either download the tar files and get the processed groundtruth files from the authors, or use the ones we provide in the repo: https://github.com/rpng/open_vins/tree/master/ov_data/euroc_mav

Hope this helps.

setareh-soltanieh commented 3 months ago

Hi @goldbattle, It worked. Thanks a lot!