Manual calibration ignored for OpenNI, OpenNI2 and Freenect drivers

[matlabbe]

Calibration files (yaml) for these drivers in camera_info folder are ignored in their init() function to use the factory calibration/registration (see CameraRGBD.cpp). Having the option (like with Freenect2) to use manual calibration, at the cost of doing software registration, may be benefic in some cases if we want more precision than the factory calibration.

[antoniodepth]

That would be really helpful. Factory calibration makes our models are curved.

[matlabbe]

Note that we cannot re-compute the depth values for structured-light sensors, as we don't know the IR pattern and the algorithm used to compute the disparity: http://wiki.ros.org/kinect_calibration/technical

ROS testing is required before trying to implement it in the standalone. See tutorials http://wiki.ros.org/openni_launch/Tutorials/IntrinsicCalibration and http://wiki.ros.org/openni_launch/Tutorials/ExtrinsicCalibration

Note that depth calibration #114 may help more to decrease distortions.

[antoniodepth]

Of course, I have made a previous calibration with ROS. I only need to apply it.

[matlabbe]

In ROS, openni2_launch, openni_launch and freenect_launch use the calibration only for the published camera info, it is not used to rectify/register again the images received by OpenNI, OpenNI2 and Freenect drivers. What I can see is that we still keep the raw images from the sensor, but the camera_info is different, so this may affect the scale of the point cloud on reconstruction but not the distorsion problem.

Without needing to rectify/register again the images, this can be implemented relatively easily then. EDIT ROS does software rectification/registration with the calibration published by drivers above, follow "sw_registered" in rgbd_launch.

[antoniodepth]

Ok, thank you for your help. I hope that you could implement it to use our manual calibration in Freenect2 soon.

[matlabbe]

For Freenect2, it is already implemented.

When starting Freenect2 before calibration:

[ WARN] (2016-09-13 12:13:59) CameraModel.cpp:267::rtabmap::CameraModel::load() Could not load calibration file "C:\Users\Mathieu\Documents\RTAB-Map/camera_info/506816242542_left.yaml".
[ WARN] (2016-09-13 12:13:59) CameraRGBD.cpp:1247::rtabmap::CameraFreenect2::init() Missing calibration files for camera "506816242542" in "C:\Users\Mathieu\Documents\RTAB-Map/camera_info" folder, default calibration used.

When starting Freenect2 driver after calibration (when default Depth2Color is still selected):

[ WARN] (2016-09-13 12:13:27) CameraRGBD.cpp:1254::rtabmap::CameraFreenect2::init() Freenect2: When using custom calibration file, type kTypeColor2DepthSD is not supported. kTypeDepth2ColorSD is used instead...

[matlabbe]

Calibration for OpenNI-PCL and OpenNI-CV drivers won't be implemented, as the interfaces (pcl::OpenNIGrabber and cv::VideoCapture) don't let us choose between registered and unregistered depth.

[antoniodepth]

I'm using the Xtion Pro Live (with OpenNI2). That means that could I use it now?

[matlabbe]

Yes, if you click Calibrate in Preferences->Source with OpenNI2 or Freenect drivers, a 3-steps calibration would start. It would be also possible to copy calibration files from ROS calibration into RTAB-Map's camera_info folder, with some renaming:

########_rgb.yaml
########_depth.yaml
########_pose.yaml

where ######## is the unique serial/name of the sensor. Note that you may get a little better RGB-Depth registration, but this doesn't fix depth distorsions... follow #114 for that.