search

PRBonn / semantic-kitti-api

SemanticKITTI API for visualizing dataset, processing data, and evaluating results.
http://semantic-kitti.org
MIT License
737 stars 184 forks source link

Questions about the poses.txt in SemanticKITTI #39

Closed jingyibo123 closed 4 years ago

jingyibo123 commented 4 years ago

Hello,

For the official KITTI dataset, groundtruth poses were given, which I personally assume were generated from GPS/IMU results ?

The SemanticKITTI dataset also provided poses.txt, which

contain the manually looped-closed poses for each capture (in the camera frame) that were used in the annotation tools to aggregate all the point clouds.

Can I ask if the generation of these poses include usage of the IMU/GPS raw data? Or merely via lidar/camera sensor odometry + manual loop closure?

Thanks a lot for answering.

jbehley commented 4 years ago

The poses for labeling and that we published are generated with our surfel-based SLAM approach which includes loop closures. Only on two sequences, we had to manually add loop closures. We do not use the KITTI poses. However, we still use the convention that the poses are represented in the camera coordinate system and therefore need to convert it via the calibration file to the LiDAR coordinate system.

jingyibo123 commented 4 years ago

Thanks for the response! I asked the question because I was surprised to see worse point cloud quality(in terms of multi-scan aggregation) from the KITTI groundtruth poses than the result I reproduced with Suma. The following scene is from sequence 01, above is KITTI gt poses, below is Suma poses..

Screenshot from 2020-03-30 15-53-18

Screenshot from 2020-03-30 15-53-05

The quality of multi-scan aggregation should be a direct indicatoro of relative pose accuracy, does that suggest that the "ground truth" poses provided by KITTI is not so perfect?

jbehley commented 4 years ago

We know that for loop closure regions the ground truth trajectory is not consistent. We also know that for some sequences the ground truth is obviously wrong due to supposedly a wrong initialization of the IMU (sequence 08).

However, we have to account for the fact that we are using the "descewed" scans. It also might be that the IMU drifted here more. We simply don't know, what causes here the "blurry" results.

But that's the main reason, why we took estimated poses rather then simply the ground truth.

jingyibo123 commented 4 years ago

Then the way I see it, the KITTI odometry evaluation and ranking could be quite biased, at least in terms of mid-range(200m) relative pose accuracy. Congratulations though on the great work with SuMa !

Edit this thread discusses the flaws of KITTI dataset.