MichaelGrupp / evo

Python package for the evaluation of odometry and SLAM
https://michaelgrupp.github.io/evo/
GNU General Public License v3.0
3.47k stars 753 forks source link

Alignment when calculating APE #268

Closed YilmazKadir closed 4 years ago

YilmazKadir commented 4 years ago

Hello, I want to ask you why we align the ground truth and the estimation sequences using the whole trajectory when calculating APE. It makes sense to align the first poses, however aligning the whole sequence seems to be cheating. Because we do not know how much error we will make at the beginning. I know it is optional to align the whole trajectory, but in ORB SLAM 2 paper the results for APE is calculated aligning the whole trajectory and I do not understand why we would do that.

MichaelGrupp commented 4 years ago

It all depends on what you want to measure. A large error that only happens at the beginning can also be detected as an outlier in the RPE. And it's not cheating if you align all the trajectories you compare, as the APE will still be better if the trajectory is better. But yes, if that's not how you want to measure you can also leave any alignment out or only align the origin etc.

I would recommend to take a look at the papers of Rainer Kümmerle and Jürgen Sturm, they describe the problems with bias of errors at beginning/end and when alignment makes sense. They're also listed at the bottom of this page: https://github.com/MichaelGrupp/evo/blob/master/notebooks/metrics.py_API_Documentation.ipynb