Difference in maximum APE using a full and a cropped trajectory

[WongRZ]

Hi, dear author, Firstly, thank you very much to share this code. It helped me a lot. I have a problem. when i use evo_ape to evaluate a result from full trajectory of EuRoC V202 dataset and a part of the same trajectory, they both have the max error, but one is 0.670081 and another is 0.646771. these two max errors should be same one because they came from the same trajectory, but in this case ,they are different.

[WongRZ]

May be you can test in your trajectory data

[MichaelGrupp]

Please provide all data and steps how to reproduce (the issue template is your friend :wink: ).

[WongRZ]

OK, thanks, dear author. My evo version is v0.9.8.6 and my system is Ubuntu 16.04. I didn't change the source code. These two files are my testing data. The first one v202_orign.txt is a full trajectory file and another one v202_part is from row 50 to row 450 of v202_orign. Then i used two commands to test them. The "Console output:" that i just pasted the error.

Command:

# evo_ape euroc V202_groundtruth.csv v202_orign.txt  -va --plot --save_results results/v202_orign.zip

Console output:

# APE w.r.t. translation part (m)

       max  0.670081
      mean  0.155985
    median  0.112952
       min  0.007594
      rmse  0.196911
       sse  25.358196
       std  0.120178

Command:

# evo_ape euroc V202_groundtruth.csv v202_part.txt  -va --plot --save_results results/v202_part.zip

Console output:

# APE w.r.t. translation part (m)

       max  0.628086
      mean  0.178977
    median  0.125446
       min  0.023779
      rmse  0.217137
       sse  21.264007
       std  0.122946

v202_part.txt v202_orign.txt

As you can see from the two result trajectory images. They both have the max error part, but their values are different. They should be the same, i think.

[WongRZ]

Sorry, the result image of v202_part is this one.

[WongRZ]

And the groundtruth file called V202_groundtruth.csv is "data.csv" from folder named "state_groundtruth_estimate0" of EuRoC MAV ”Vicon Room 2 02“ dataset.

[MichaelGrupp]

That's pretty much expected behavior if you use least-squares Umeyama alignment. The alignment transformation of the shorter trajectory is slightly different, which in turn leads to a different APE.

You can visualize this with:

evo_traj tum --ref groundtruth.tum v202_* --align --plot

Regardless of alignment, the result will of course also be different if you cut out the part with the maximum error.

[WongRZ]

Thanks a lot, dear author. I'm probably getting it. So if I want to compare two trajectories, their length should be the same?

[MichaelGrupp]

No. But if you cut parts of a trajectory off, you can't expect the result to be the same.

(of course it usually makes sense to compare trajectories of approximately the same length, but that's a different story)