search

microsoft / Azure_Kinect_ROS_Driver

A ROS sensor driver for the Azure Kinect Developer Kit.
MIT License
304 stars 226 forks source link

camera intrinsics calibration #200

Closed christian-rauch closed 3 years ago

christian-rauch commented 3 years ago

Fixes #105, #199

Description of the changes:

Required before submitting a Pull Request:

I tested changes on:

With the standard monocular calibration described in http://wiki.ros.org/camera_calibration/Tutorials/MonocularCalibration can now achieve a much better AprilTag to point cloud alignment:

atag_kinect_calib1 atag_kinect_calib2

You have to use the new kinect_rgbd.launch launch file to rectify the images.

Note: To calibrate the IR sensor, you likely have to reduce the scaling to something around 0.2

roslaunch azure_kinect_ros_driver kinect_rgbd.launch ir_mono8_scaling_factor:=0.2

to prevent saturation of the IR image.

christian-rauch commented 3 years ago

Some more information about the different intrinsic parameters. I am using the kinect_rgbd.launch launch file which uses rgbd_launch for rectification and registration. The raw images are published to /rgb/image_raw and /ir/image_raw (as before) and the rectified images are published to /rgb/image_rect_color and /ir/image_rect_ir.

The ROS image_proc nodelets can use the factory calibration for rectification and registration, but the factory and manual calibration differs in how the RGB and IR images are warped. The factory calibration fully warps the image such that all lines are straight but adds an irregular black border to the image due to the full warping. The manual calibration only partially warps the image such that the borders are kept. Hence, the manual calibration is useful if you intend to post-process the rectified image (e.g. edge and keypoint detection) and do not want to deal with manual filtering of the output. Of course, manual calibration will also be required if you intend to change the optics of the camera.

The following raw image (/rgb/image_raw): image2_raw

will be rectified with the factory calibration (/rgb/image_rect_color) as: image2_rect_factory

and the manual calibration will generate: image2_rect_manual

amelhassan commented 3 years ago

I also verified using Melodic on Windows that this functions, thanks for your contribution @christian-rauch. Do you know if its possible to have something similar for ROS2 (it doesn't look like the rgdb_launch package has a ROS2 equivalent)? Also it would be helpful if you included some of your quick comments on using the kinect_rgbd.launch in the using.md (when to use it, and mention needing the rgdb_launch package).

christian-rauch commented 3 years ago

I added documentation for the launch files and calibration to usage.md.

A quick look shows that there is at least an active ros2 branch for the camera_info_manager and for the image_proc that does the rectification. That should cover the basic calibration and rectification features. I haven't looked into implementing this for the ROS2 as our setup does not use ROS2 yet. Maybe someone at Microsoft can adapt this to the ROS2 version of the driver?

amelhassan commented 3 years ago

Great, thank you for those changes. Our team will look into creating a similar change for the ROS2 version. I created issue #211 to represent that.

bmegli commented 2 years ago

The ROS image_proc nodelets can use the factory calibration for rectification and registration, but the factory and manual calibration differs in how the RGB and IR images are warped. The factory calibration fully warps the image such that all lines are straight but adds an irregular black border to the image due to the full warping. The manual calibration only partially warps the image such that the borders are kept. Hence, the manual calibration is useful if you intend to post-process the rectified image (e.g. edge and keypoint detection) and do not want to deal with manual filtering of the output. Of course, manual calibration will also be required if you intend to change the optics of the camera.

For the record:

It should be relatively easy to: