[BUG] {RGB color camera_info is not correct?}

[andraspalffy]

TL; DR Looking for the correct intrinsic params of Oak d pro poe. I do not think that the one linked below is the one.

Describe the bug

Hello, I have a Oak d pro poe, and I would like to project LiDAR point cloud into to the RGB color camera image.

I use this launch file just to have an RGB camera: https://github.com/luxonis/depthai-ros/blob/noetic/depthai_examples/launch/rgb_publisher.launch, which publishes rgb_color/image/compressed and rgb_color/camera_info topics (among others).

The latter is filled out by thes infos: https://github.com/luxonis/depthai-ros/blob/e544cb9abf2c23b3dda19fd3bee4df69564dcca9/depthai_examples/params/camera/color.yaml

However, adding the camera topic in Rviz as a "camera view" which uses the camera_info topic, lidar, and other markers are projected badly. Of course, extrinsic calibration could be the reason, but it is not, the offset is not consistent along the image (i.e. it is not always to the left, or always up, more like a zoom/distortion error).

Minimal Reproducible Example Launch the launch file above with the given camera, and project a a single marker onto an rviz cam panel.

Expected behavior What I expect to happen is that using the "official" camera_info, my points are projected onto the camera in a way that can explained by extrinsic calibration error.

Example image: As you can tell, the LiDAR pointcloud is way bigger and off in both directions (deep read on the left should be inside the door).

[Serafadam]

Hi,

1. rgb_publisher.launch is deprecated and will be removed in next release. We recommend either using camera nodelet in depthai_ros_driver or stereo_inertial_publisher as entrypoints (depthai-ros-driver's camera is the "default" driver, while nodes in depthai_examples showcase how one can develop their own custom solution with custom publishers.
2. Camera calibration files are used here as an example only to showcase external calibration loading. Each camera has a unique calibration that is flashed in a factory. If no calibration file is specified to the ROS nodes, then device calibration is used, which will be more accurate for each camera. If you notice that there is something wrong with the device calibration, please let us know.

[andraspalffy]

Thanks, that could be very much it! Can you specify how to not specify the calibration file to the ros node?

[Serafadam]

Hi, both stereo_inertial_publisher and camera node publish camera's internal calibration by default.

[andraspalffy]

Thank you, it worked!