How can I combine a realsense d455 camera with a lidar ?

[Althyrios]

I have a gazebo-ROS2 simulation in place using lidar and a rgbd realsense d455 camera. At the moment, when I run rtabmap and using icp_odometry, I get a point cloud corresponding to the lidar view. If I use rgbd_odometry from the code found in the following example https://github.com/introlab/rtabmap_ros/tree/ros2/rtabmap_examples/launch I get a point cloud that corresponds to what the camera sees. How can I merge the two point clouds into one?

[matlabbe]

It depends on which topic you are looking at. If you look at the /rtabmap/cloud_map topic, yes, it is possible. Use parameter `Grid/Sensor: '2``. The other way would be to combine the point clouds before rtabmap node, and feed the combined point cloud like it was a single 3d laser scan input.

[Althyrios]

Hello,

Thank you for your prompt reply.

First of all, I would like to provide more information about my problem.

I use a 3D LiDAR (Livox) and an RGB-D Camera (RealSense D-455) and I would like to map / generate a point cloud of an area using both sensors. I would like to use ICP odometry with the LiDAR data, and mapping using both the 3D point cloud generated by the LiDAR and the 3D point cloud generated from the Realsense D455 RGB-D camera. The main idea is to have better localization using ICP odometry with LiDAR data, and combine the RGB-D camera to generate a fused point cloud RGB information (colored point cloud, generated using both LiDAR and Camera).

My colleague is working on the same problem, and you can find more information and context here: http://official-rtab-map-forum.206.s1.nabble.com/Combine-3D-Lidar-and-RGB-D-Camera-to-create-a-single-3D-point-cloud-sensor-fusion-td10162.html

Feel free to reply to him directly, as he is working on this for a while now.

Also, I'm a new rtabmap user, could you please explain how the Grid/Sensor parameter works and how to use it?

Thanks in advance for any help you can give me. (modifié)