MichaelGrupp / evo

Python package for the evaluation of odometry and SLAM
https://michaelgrupp.github.io/evo/
GNU General Public License v3.0
3.39k stars 745 forks source link

Question about Run a metric on trajectories #204

Closed lixz123007 closed 4 years ago

lixz123007 commented 5 years ago

**Description:

I got the poses from rtabmap(orb odometry mode and rtabmap odometry mode),and kitti groud truth.When i run

evo_traj kitti KITTI_00_ORB.txt KITTI_00_SPTAM.txt --ref=KITTI_00_gt.txt -p --plot_mode=xz

It's OK ,and good. (only a little problem,but it is not important).But when i want run this :

evo_ape kitti KITTI_00_gt.txt KITTI_00_ORB.txt -va --plot --plot_mode xz --save_results results/ORB.zip

I got

Loaded 4541 poses from: 00.txt Loaded 454 poses from: poseskitti25Gunorbok.txt

Aligning using Umeyama's method... [ERROR] data matrices must have the same shape

So i search this question and see the two questions like this .So i download "kitti_poses_and_timestamps_to_trajectory.py" file .And add timestamps to my kitti groud truth file(which is goundtruth.txt) Then i don't know how to continue and got a lot of errors. When i run:

# evo_ape kitti goundtruth.txt KITTI_00_ORB.txt -va --plot --plot_mode xz --save_results results/ORB.zip

I got

[ERROR] KITTI pose files must have 12 entries per row and no trailing delimiter at the end of the rows (space)

When i run:

# evo_ape tum groundtruth.txt KITTI_00_ORB.txt -va --plot --plot_mode xz --save_results results/ORB.zip

I got

[ERROR] TUM trajectory files must have 8 entries per row and no trailing delimiter at the end of the rows (space)

When i run:

# evo_traj kitti KITTI_00_ORB.txt --ref=groundtruth.txt -p --plot_mode=xz --correct_scale --align

I got

[ERROR] KITTI pose files must have 12 entries per row and no trailing delimiter at the end of the rows (space)

So i don't knew how to do next!Maybe you have answered in other quesions. But maybe i'm too noob to understand it .

I upload those txt in: https://github.com/lixz123007/question

00.txt: origin kitti ground truth . groundtruth.txt: the ground truth i turnned. posekitti25GORBOK.txt and poseskitti25Gunorbok.txt :the poses from rtabmap(orb odometry mode and rtabmap odometry mode). times: origin kitti timestamps.

I convert my files to the 'tum' too.But it comes a problem.The original groundtruth have 4541 rows.But my files only 454 rows.So i delete times to 454 rows. Althouth evo_ape tum groundtruth.txt KITTI_00_ORB.txt -va --plot --plot_mode xz --save_results results/ORB.zip works,but i think it isnot correct because of different timestamps! Here is the picture picture2

UserUsingGit commented 5 years ago

Hi, I think there are a few things to correct here. The error "data matrices must have the same shape" appears because you have a different number of entries in your ground truth file and your SLAM result file. This is only a problem using the option "kitti" in evo_ape or evo_rpe. With the script "kitti_poses_and_timestamps_to_trajectory.py" you can convert your files to the "tum" format, and I guess youll need to apply it to both (your ground truth data and your SLAM results).

evo_ape kitti KITTI_00_gt.txt gourdtruth.txt -va --plot --plot_mode xz --save_results results/ORB.zip Here you are comparing ground truth to converted ground truth, which makes no sense. Also, I would call the ground truth file "groundtruth.txt" and not "groudtruth.txt". In the above command youre writing "gourdtruth.txt", which is neither of those. And in your last command, youre writing: evo_traj kitti KITTI_00_gt.txt --ref=groundtruth.txt -p --plot_mode=xz --correct_scale --align But groundtruth.txt is not the name of your file.

Anyway, you have to apply the commands to the same data formats, so if youre using "evo_ape tum ...", you have to convert both the groundtruth file and the SLAM result into the "tum" format.

lixz123007 commented 5 years ago

I'm really sorry about my english problem, and i have corrected them in my question. I convert my files to the 'tum' too.But it comes a problem.The original groundtruth have 4541 rows.But my files only 454 rows.So i delete times.txt to 454 rows . Althouth evo_ape tum groundtruth.txt KITTI_00_ORB.txt -va --plot --plot_mode xz --save_results results/ORB.zip works,but i think it is not correct because of different timestamps! Here is the picture picture2

UserUsingGit commented 5 years ago

I cannot see the picture unfortunately, can you upload it directly to Github? As far as I know, there should be no problem comparing trajectories with varying amount of rows using "evo_ape tum". I have not tried it though, so I have no experience with that. Make sure both files have 8 columns (timestamp x y z qx qy qz qw). Is your KITTI_00_ORB.txt converted?

lixz123007 commented 5 years ago

I upload 3 picture in my: https://github.com/lixz123007/question . The groundtruth have 4541 rows and my KITTI_00_ORB only have 454 rows.Both of they don't have timestamps . And i find timestamps file(4541 rows) of groundtruth and converted it.But if i want converted my KITTI_00_ORB ,i have to delete to 454 rows. If i just delete reset 4087 rows just leave first 454 cows,the result is picure: error1 in my github. If i delete 9 rows in every 10 rows i will get 455 rows .And if i delete last cows, the result is better in picture :error3. But if i leave the last rows ,and delete a random rows ,the result is in picture : error2. I'm not sure whether you understand my means or not: If i change the timestamps ,i will get different result which make me think it is not normal . Is it the problem of my parameter ?

MichaelGrupp commented 5 years ago

If it is possible, I think the easiest solution would be to save the trajectory of the SLAM algorithm (that you want to evaluate) in a trajectory file with timestamps, not as "KITTI". Then you can compare it to the groundtruth trajectory (the one you got with kitti_poses_and_timestamps_to_trajectory). Different numbers of poses shouldn't be a problem then because the poses will be associated via timestamps.

Deleting random rows won't help you here.

lixz123007 commented 5 years ago

I can save the poses with timestamps, But it seems have 3 kind of timestamps .The timestamps of groundtruth is "1.037359e-01".But the timestamps of poses saved by my slam is "2018-03-10 13:29:04.130157512"or "1520659744.130157512" .I even don't know whether the first kind of timestamps can become second two kind of timestamps. IF i force two kind of timestamps to "evo_ape tum" it will be a error.

MichaelGrupp commented 4 years ago

The timestamps must match. I recommend to check if you can change the SLAM software to use the timestamps of the dataset.