Issue about `create_color_image_mapped_to_depth` and `create_depth_image_mapped_to_color`

[kunlin596]

Hey guys,

I've got a RealSense SR300 and I was trying to use RealSense SDK to create a fusion image which used the code below approximately.

intrinsics color_intrin = convert_intrinsics(device->get_stream_intrinsics(rs::stream::color));
intrinsics depth_intrin = convert_intrinsics(device->get_stream_intrinsics(rs::stream::depth));
extrinsics extrin = convert_extrinsics(device->get_extrinsics(rs::stream::depth, rs::stream::color));

// ...
// Got the color image and depth image using the code about the same as in the projection sample.
// ...

// Using the functions below to create fusion image.
auto colorImageMappedToDepth = get_unique_ptr_with_releaser(projection_->create_color_image_mapped_to_depth(depthImage.get(), colorImage.get()));
auto depthImageMappedToColor = get_unique_ptr_with_releaser(projection_->create_depth_image_mapped_to_color(depthImage.get(), colorImage.get()));

I tried to print out extrin's rotation and translation components and it seemed valid. But soon after I created both of the two mapped images, I found out that the translation parameter in the extrinsic is weird. By which I mean either color image or the depth image is shifted horizontally -ish and the mapped images are not in the right place.

I tried to extrin.translation[0] *= 10 to enlarge the x component of translation, and this fixed part of the problem and it seemed almost right but still shifted (after all it's just a work around).

I don't know whether the parameters fixed for every SR300 or it's read dynamically from every hardware and values are all different. But either way I think something is wrong about the extrinsic parameters, or the way I took it.

Any ideas about this situation?

[andrey-golubev]

Hi Kun Lin,

Sorry for the late response. I am currently not completely sure what the problem is. You can try to use rs::stream::rectified_color instead for now, shift should disappear with this work around. We will come back to you once we figure out the issue and have a proper answer for you.

Meanwhile, I would recommend you to ask about "weirdness" in translation component on the librealsense page: as we simply pass intrinsics and extrinsics data provided by librealsense to our domain (without any additions, as far as I know).

By the way, the SDK do not officially support SR300 camera, only ZR300 (you can see this information in the "Compatible Devices" section in the README.md on the main page).

Thanks for you question!