LiDAR-to-camera calibration

[richardmarcus]

Hi, thanks for providing this dataset.

I have a question about the calibration files you provide (cam_to_cam, imu_to_lidar and lidar_to_cam). Is this the result of the calibration process you describe in the paper, including the RANSAC based optimisation for lidar to camera calibration?

And followup questions: Are the camera images already rectified?

Can, like in KITTI, a point X in lidar coordinate be projected to the rgb_camera as

Y = P_rect_02 Rrect?? (R|T)_lidar_to_cam * X

Is this correct? Which is the reference camera coordinate system?

[l1997i]

Hi @richardmarcus, thank you for your interest in our dataset and for your questions!

Calibration Files

Yes, the calibration files we provide (cam_to_cam, imu_to_lidar, and lidar_to_cam) are the result of the calibration process described in our paper, including the RANSAC-based optimization for lidar-to-camera calibration.

Camera Images Rectification

Yes, the camera images are already rectified. You can directly use the provided projection matrices for your applications.

Point Projection

You are correct that a point X in the lidar coordinate can be projected to the RGB camera using that equation. The reference camera coordinate system is typically the main front camera (similar to KITTI’s camera 02).

[richardmarcus]

Hi, thanks for getting back to me. When using the given calibration data like this, I never managed to get a good alignment between camera and LiDAR. Maybe I still did something wrong, but it seemed to me that one of the calibration files is not the correct one or something like that.

For my research, I had success with https://github.com/koide3/direct_visual_lidar_calibration for recalibration. calib.json Possibly, this is helpful for others who run into the same problem.

[l1997i]

Hi @richardmarcus, thank you for getting back to me and sharing your experience.

The calib.json file you provided appears to be on the KITTI dataset. Have you tested the direct_visual_lidar_calibration solution on our DurLAR dataset? direct_visual_lidar_calibration was not yet published when the DurLAR dataset was released. If you have not tested it on DurLAR, I will attempt to use it to further verify our calibration results and let you know.

[richardmarcus]

The calib.json was generated by me with direct_visual_lidar_calibration on DurLAR, I simply had not changed the naming as I had used a KITTI reference to figure out how to get the ROS format right

[l1997i]

The calib.json was generated by me with direct_visual_lidar_calibration on DurLAR, I simply had not changed the naming as I had used a KITTI reference to figure out how to get the ROS format right

The calibration result looks good based on the visualization. I will further verify the calibration result and update our calibration file as needed. Thank you very much for your feedback and help.

[ilay-chen]

Hi, There is a rosbag file that can be downloaded? Or how can I transfer the data to rosbag? I had many difficulties with https://github.com/tomas789/kitti2bag because the format is different... Maybe you have some code to translate it to kitti format? (the gps+imu, timestamps etc)

Thank you

[richardmarcus]

Hi, I also used kitti2bag, but for direct_visual_lidar_calibration I only needed a few lidar scans and camera images, so I kept the kitti folder structure and copied the data. (If you want to only get the calibration for DurLAR and have no further calibration applications, I would recommend to try my calib.json file because there are a few obstacles to get results)

[ilay-chen]

There is no need of the imu and GPS for direct_visual_lidar_calibration? This is my main kitti2bag issue now... I need to do the calibration my self for my research... Can you elaborate about your obstacles and how you success? Thank you!

Maybe there is close to translate the imu and GPS to oxts kitti syle?

[richardmarcus]

I don't think you need imu and GPS, the algorithm creates a LiDAR reference from multiple scans via CT-ICP and takes just one camera image. Some things I remember:

• do not use the first few camera frames if the exposure has not yet settled (e.g. for DurLAR)
• consider using another ICP algorithm so that you have more control for the number of frames used, potentially even with support for motion correction/deskewing, and then just provide your merged point_cloud as lidar frame
• you may need the intrinsics and distortion (in case of DurLAR fill it with zeros because camera images are undistorted already)

[l1997i]

We conducted a targetless calibration using the aforementioned method. We have updated the arxiv version of our paper (appendix).

Additionally, we have included new calibration results and the ROS bag used for the calibration (refer to https://github.com/l1997i/DurLAR?tab=readme-ov-file#get-started). We have also updated the README file to explain and clarify our changes. Hope this will help.