Is extrinsic_R in avia_resize.yaml must be identity?

[kekeliu-whu]

Thank you for your great contribution!

I am testing hilti2021 dataset on FAST-LIVO which contains a livox mid70, an embeded IMU and a 1440x1080 10Hz global shutter camera.

I find that when I write really extrinsic parameters on the left which extrinsic_R is not identity, FAST-LIVO will result in bad initialization showing that image is not aligned with livox scan.

So I make some changes to force lidar and imu into one frame:

1. Add some codes in livox_scan_cbk() to transform livox points from lidar frame to IMU frame;
2. Set paramter to the right above. Then the image align well with livox scan.

I would like to know if extrinsic_R in avia_resize.yaml must be identity?

[xuankuzcr]

Thanks for your interest and testing! The extrinsic_R in avia_resize.yaml doesn't have to be an identity matrix. Rcl represents the rotation matrix of LiDAR frame w.r.t. Camera frame. Pcl represents the translation of LiDAR frame w.r.t. Camera frame. extrinsic_R represents the rotation matrix of LiDAR frame w.r.t. IMU frame. extrinsic_T represents the translation of LiDAR frame w.r.t. IMU frame. If you set them correctly according to the physical meaning, there should be no wrong alignment of the image and LiDAR scan. Additionally, you need to change the camera's projection model to adapt to the Hilti dataset.

[kekeliu-whu]

Thanks for your answers!

[nuyhead-hwang]

@xuankuzcr

I also tried to test the HILTI 2021 dataset.

But trajectory which is estimated by FAST-LIVO very unstable and eventually diverged.

I set the parameters provided here as shown in the code below. but FAST-LIVO diverge during the dataset playback.

feature_extract_enable : 0 point_filter_num : 4 max_iteration : 3 dense_map_enable : 1 filter_size_surf : 0.3 # 0.3 filter_size_map : 0.3 # 0.4 cube_side_length : 1000 debug : 1 grid_size : 40 patch_size : 8 img_enable : 1 lidar_enable : 1 outlier_threshold : 50 ncc_en: true ncc_thre: 0.5 img_point_cov : 1000 laser_point_cov : 0.001 cam_fx: 696.7174426776 cam_fy: 696.4862496732 cam_cx: 708.4206218964 cam_cy: 535.6712007522 common: lid_topic: "/os_cloud_node/points" imu_topic: "/os_cloud_node/imu" preprocess: lidar_type: 3 # Ouster scan_line: 64 blind: 1 # blind x m disable mapping: acc_cov_scale: 100 #10 gyr_cov_scale: 10000 #10 fov_degree: 180 extrinsic_T: [ 0.0, 0.0, 0.0] extrinsic_R: [ 1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0] camera: img_topic: /alphasense/cam1/image_raw Rcl: [0.0, 0.0, 1.0, -1.0, 0.0, 0.0, 0.0, -1.0, 0.0] Pcl: [0.054, 0.137, -0.040]

And there is no part where the lens distortion is processed in the code. why is that ?

And when the 3D point cloud of the rotating LiDAR is re-projected onto the image, how to deal with rear area pointclouds that are not visible in the image ?

[Camilochiang]

Thanks for your interest and testing! The extrinsic_R in avia_resize.yaml doesn't have to be an identity matrix. Rcl represents the rotation matrix of LiDAR frame w.r.t. Camera frame. Pcl represents the translation of LiDAR frame w.r.t. Camera frame. extrinsic_R represents the rotation matrix of IMU frame w.r.t. LiDAR frame. extrinsic_T represents the translation of IMU frame w.r.t. LiDAR frame. If you set them correctly according to the physical meaning, there should be no wrong alignment of the image and LiDAR scan. Additionally, you need to change the camera's projection model to adapt to the Hilti dataset.

your translation values in your configuration file have the oposite symbols that the documentation from livox. The imu is behind and above from the Lidar, so I was asking me: isn't this values actually translation and rotation lidar w.r.t imu?

[xuankuzcr]

Thanks for your interest and testing! The extrinsic_R in avia_resize.yaml doesn't have to be an identity matrix. Rcl represents the rotation matrix of LiDAR frame w.r.t. Camera frame. Pcl represents the translation of LiDAR frame w.r.t. Camera frame. extrinsic_R represents the rotation matrix of IMU frame w.r.t. LiDAR frame. extrinsic_T represents the translation of IMU frame w.r.t. LiDAR frame. If you set them correctly according to the physical meaning, there should be no wrong alignment of the image and LiDAR scan. Additionally, you need to change the camera's projection model to adapt to the Hilti dataset.

your translation values in your configuration file have the oposite symbols that the documentation from livox. The imu is behind and above from the Lidar, so I was asking me: isn't this values actually translation and rotation lidar w.r.t imu?

I'm sorry, I made a mistake. It has been corrected.