search

minwoo0611 / MA-LIO

Asynchronous Multiple LiDAR-Inertial Odometry using Point-wise Inter-LiDAR Uncertainty Propagation
GNU General Public License v2.0
287 stars 36 forks source link

Reproducing results in Hilti SLAM Dataset 2021 #19

Closed KkCabin closed 5 months ago

KkCabin commented 5 months ago

Hello, @minwoo0611, Thanks for the great work! I'm trying to reproduce results in TABLE II in the paper. However I ran into some problems. During testing I found that for the sequences Basement, Campus, and Construct, the ground-truth provided on the dataset's official website were incomplete and missing a lot of trajectory information. I am wondering how did you do the evaluation about these sequences in TABLE II? Thanks for any advices!

minwoo0611 commented 5 months ago

Hello @KkCabin,

I apologize for the delay in my response. I had some scheduling conflicts last week, but I'm here now to address your question.

Regarding the evaluation of sequences Basement, Campus, and Construct in TABLE II of the paper, I understand your concern about the ground-truth data provided on the dataset's official website. The ground-truth for the Hilti 2021 dataset is obtained using Ground Control Points (GCP), which results in sparse ground-truth data. It's important to note that this ground-truth does not include the full trajectory information. However, GCPs are known for their accuracy in terms of position, making them suitable for SLAM evaluation.

For more detailed information about the use of GCPs and the dataset's ground-truth, I would recommend referring to the paper itself. You can access the paper at the following link: LiDAR Odometry Survey. This paper contains the evaluation method for SLAM, and it might provide additional insights into how the evaluation was conducted for these specific sequences.

Additionally, you can find the evaluation tool for the Hilti 2021 dataset in their repository, which can be accessed at the following link: Hilti 2021 GitHub Repository. We utilized this repository when validating MA-LIO and creating the table for MA-LIO's performance.

I hope this information helps you with your efforts to reproduce the results in TABLE II of the paper. If you have any further questions or need additional assistance, please feel free to ask.

Thank you!

KkCabin commented 5 months ago

Hello, @minwoo0611, Your reply helped me a lot, thank you sincerely. By the way, MA-LIO is a very good job, thanks for your efforts and open source spirit.