search

snail-radar / dataset_tools

loader for the generic 4D radar dataset
https://snail-radar.github.io
BSD 3-Clause "New" or "Revised" License
14 stars 1 forks source link

Reference trajectories #3

Closed Dcasadoherraez closed 2 weeks ago

Dcasadoherraez commented 3 weeks ago

Hello! Thanks so much for this nice dataset! It seems really well organized and the data seems good so far :) I have one question regarding the reference trajectories.

This is a plot of all the tls_T_xt32.txt files in ref_trajs. image

Those plots seem to only belong to this region of the campus: image

Meanwhile, the info arts and eng. and the info arts faculties don't belong to these paths: image

I guess the missing parts of the long trajectories are due to this statement: ''For large-scale sequences, since the TLS map only covers the beginning and end parts of the sequence, we generate the reference trajectory only for the beginning and end subsequences within the TLS coverage, as done in TUM-VI Schubert et al., 2018.''

Using utm50r_T_x36dimu.txt does return the entire trajectory of the sequence image

Do you suggest any method for evaluation based on tls_T_xt32.txt, without having the entire ground truth trajectory? Have you noticed any variations in accuracy compared to utm50r_T_x36dimu.txt?

Thanks in advance!!!!

JzHuai0108 commented 3 weeks ago

Thank you for your interest and viz effort in our dataset.

Indeed our accurate reference trajectories only cover the campus area due to the limited TLS coverage. For the accuracy evaluation with tls_T_xt32.txt, the classic ATE RMSE can be used as implemented in rpj_traj_eval_tool since it does not require a small time gap between consecutive poses. We provide the utm50r_T_x36dimu.txt for the purpose of place recognition which usually does not require accurate height values. Meanwhile, we are working to provide the complete trajectories by fusing the TLS trajectories, lidar odometry, and the GNSS trajectories with a pose graph optimizer. It should be ready in one week, Aug 25 2024. But we only guarantee the visual quality of these trajectories :), and their suitability for place recognition.

ChiyunNoh commented 2 weeks ago

Hello author! @JzHuai0108 Has the revised gt-trajectory been uploaded? If it hasn't been uploaded yet, could you mention me when it has been uploaded?

JzHuai0108 commented 2 weeks ago

Thank you for your interest. I am tuning the least squares solver that fuses the odometry data. The fused solution should be ready in two or three days. I will leave a message here when it is ready to download.

Dcasadoherraez commented 2 weeks ago

As a small feedback it would be amazing if you also provide them in kitti format directly.

JzHuai0108 commented 2 weeks ago

Hi @Dcasadoherraez , Ok, I will do accordingly. Note in the KITTI format each line has 12 numbers of the first 3 lines of the 4X4 transformation matrix, we have to provide the times in another file.

JzHuai0108 commented 2 weeks ago

Hi friends, I have completed the full trajectories by PGO. They are here. Note the full trajectory has large uncertainties in z within 1 meter as shown in the below quality check images, also see the readme.md inside the zip. So the full trajs are good enough for place recognition training and testing, but not intended for odometry quality benchmarking.

horizontal_quality_check vertical_quality_check

Dcasadoherraez commented 2 weeks ago

Thanks so much for your effort providing the best data possible! Really appreciate that. Certainly the uncertainty might be a problem for odometry/slam. But at least it can provide some baseline for comparison between methods.

Thanks again!

I will close the issue and reopen if I notice anything weird :D