The command to get ATE rmse for kitti stereo from ORB_SLAM2

[linpeisensh]

Thank you for your excellent work!

A simple question: When I want to get ATE rmse for ORB_SLAM2, my command is "evo_ape kitti poses/04.txt CameraTrajectory.txt -r trans_part -as" (align and correct scale).

Since the result with "-a" and "-s" is much bigger than that in paper for sequence 04 and "-as" is close to in paper, I think "-as" is correct. However, for sequence 01, 02, the result with "-as" is much smaller than that in paper. I am confused.

Could someone tell me whether my command is correct? If no, what is the correct command? (By the way the command for ape)

Thanks!

[MichaelGrupp]

Could someone tell me whether my command is correct? If no, what is the correct command? (By the way the command for ape)

Did you check what the paper says about the absolute error? RMSE of the absolute error following the ATE metric defined in the TUM-RGBD paper. Formally, the evo equivalent for this would be the RMSE of evo_ape ... --align.

Whether this will help you to replicate the paper result is a different story. You will only get exactly the same result if you use the trajectory data of the paper's author and the same evaluation script that the author used.

---

For the relative error, there was a similar discussion here: https://github.com/MichaelGrupp/evo/issues/144, with my answer here: https://github.com/MichaelGrupp/evo/issues/144#issuecomment-464330736

[surfii3z]

Hi @MichaelGrupp, thanks for the awesome package. It is much user-friendly and easier to use than other package.

I, however, have a doubt on your APE is equivalent to ATE in TUM-RGBD paper though.

In the paper, they state that

Note that the RPE considers both translationaland  rotational  errors,  while  the  ATE  only  considers  thetranslational  errors.

So I guess, they don't include rotational error but you do if I am not mistaken. In this case, how would you suggest if we only want the transitional error

[MichaelGrupp]

evo does the translational error by default. If you want measure rotational difference, you have to specify it: --pose_relation angle_deg or angle_rad.

[surfii3z]

I see, thank you @MichaelGrupp

[Johnemad96]

Thank you for your excellent work!

A simple question: When I want to get ATE rmse for ORB_SLAM2, my command is "evo_ape kitti poses/04.txt CameraTrajectory.txt -r trans_part -as" (align and correct scale).

Since the result with "-a" and "-s" is much bigger than that in paper for sequence 04 and "-as" is close to in paper, I think "-as" is correct. However, for sequence 01, 02, the result with "-as" is much smaller than that in paper. I am confused.

Could someone tell me whether my command is correct? If no, what is the correct command? (By the way the command for ape)

Thanks!

hey @linpeisensh , Did you figure out the reason behind the difference your results (using -a) and the paper's results (using -as)? thanks