StereoBM parameters for additional depth maps

[uwleahcim]

I'm trying to obtain depth images from 4 camera sets. Currently guidance is only capable of outputing 2 depth/disparity maps. Thus I'll have to use on board processing to generate the remaining 2 depth maps. I would like to know what parameters are used for the guidance to generate the depth maps, so I can generate similar maps with opencv stereoBM (to keep data consistent across the cameras).

Also I was reading to through the documentation of get_stereo_cali and noticed that get_stereo_cali does not provide distortion coefficients as the images are rectified. Assuming the cameras are calibrated with the Guidance assistant, are the left and right gray scale images obtained already rectified?

[tangketan]

For stereoBM, the maxDisparity=64, other parameters are the same as OpenCV's default. No speckle filter is used.

Yes, the output grayscale images are already calibrated. That's why we don't provide distortion coefficients.

[uwleahcim]

The default parameter for opencv stereoBM is numDisparities=0 and blockSize=21 via StereoBM::create

maxDisparity = numDisparities - minDisparties correct?

By default the blockSize is much larger than what the guidance system output. I had to tune blockSize value down to 9 to get it resemble the guidance depth output.

Ptr<StereoBM> sbm = StereoBM::create(64, 9);
smb->compute(left, right, depth);

Also, the output of stereoBM is a disparity map. Thus the colors should be inverse of the depth map.

In order to generate the depth map from disparity map I need to perform the following calculation.

depth = baseline * focal_length / disparity

depth should be a int16, so what should the units of baseline and focal_length used for the calculation? What are the units of the depth image (mm?).

If the units are in mm will my equation look like this? (247.35mm focal length and 15cm baseline)

depth = 150 * 247 / disparity

Anyways, here is the depth image I generated with the following code:

Ptr<StereoBM> sbm = StereoBM::create(64, 9);
sbm->compute(g_greyscale_image_left, g_greyscale_image_right, g_depth);
g_depth = (247.35 * 150)/g_depth;
g_depth.convertTo(depth8, CV_8UC1);
imshow(frame_id[CAMERA_PAIR_INDEX] + " depth opencv", depth8);

Opencv depth

Guidance depth

Guidance image

Seems like the scaling is just off a bit.

[uwleahcim]

When just compare the disparity maps, the images looks almost exactly the same. Now the issue is how is the guidance core computing depth map from the disparity map?

Also, when I pull disparity (either generated from opencv or guidance core) and depth from opencv (calculated based on depth = 150 * 247 / disparity) from the publish topic. The image is computer blown out white. This isn't the case with depth generated by the guidance core.

The following images were published by guidanceNode then pulled from topic via guidanceNodeTest

These depth images were generated by guidance core

These depth images were generated by opencv

[uwleahcim]

Just an update.

I'm an idiot and completely forgot about the fractional bits. Both the disparity map generated by the guidance core and opencv StereoBM are 16-bit with 4-bit fractions. Thus, one has to divide the raw disparity data by 16 to convert to floating points.

Also, the units of depth map is in meters and not mm.

Now according to guidance documentation, the depth image also has fractional bits, but it is 7-bit. Thus to match the guidance core depth image output, you have to multiple the floating point depth image by 128.

For anyone using ROS, the standard for depth images is meters in floating points. Thus you'll have to divide the guidance core's depth by 128 first to convert it into float points.

[tangketan]

Thank @uwleahcim for the explanations. Yes, Guidance has 7-bit fractions. We have given an example of displaying real depth value at the center of image in usb_example/main.cpp.