How to access the confidence stream for all the pixels of frame(piece of code to access)?

[chetansarmak]

---

Required Info
Camera Model	D400
Firmware Version	05.11.01.00
Operating System & Version	Linux (Ubuntu 16)
Kernel Version (Linux Only)	4.4.0-21
Platform	PC
SDK Version	librealsense-2.17.1
Language	C++
Segment	Robot

Issue Description

Need some help in accessing the confidence stream and to see what is the tradeoff between confidence and depth ?

[MartyG-RealSense]

There is a setting called RS2_OPTION_CONFIDENCE_THRESHOLD

A note for this setting says "The confidence level threshold used by the Depth algorithm pipe to set whether a pixel will get a valid range or will be marked with invalid range".

Dorodnic posted a code snippet on the Bountysource website that contains an example reference to RS2_OPTION_CONFIDENCE_THRESHOLD.

https://www.bountysource.com/issues/61675803-how-to-adjust-depth-image-quality-just-like-depth-quality-tools-in-my-code

[chetansarmak]

Hi Marty I have tried that piece of code

rs2::pipeline p; auto profile = p.start(); auto dev = profile.get_device(); auto sensor = dev.query_sensors().front(); sensor.set_option(RS2_OPTION_MOTION_RANGE, ...); sensor.set_option(RS2_OPTION_CONFIDENCE_THRESHOLD, ...); sensor.set_option(RS2_OPTION_FILTER_OPTION, ...);

But what values to pass for RS2_OPTION_CONFIDENCE_THRESHOLD are ranging from 0-15 and that too float value .

which throws me a error

terminate called after throwing an instance of 'rs2::invalid_value_error' what(): object doesn't support option #17

And can I get the confidence values of all pixels of the depth map for D435 camera module ?

This issue says #1950 no option of getting confidence values

[MartyG-RealSense]

There are some stereo-camera confidence map calculation approaches not specific to RealSense that people have tried. A popular method involves using what is called a 'disparity map'.

Here is an example using the OpenCV vision software (which RealSense SDK 2.0 is fully compatible with) that calculates the confidence map by computing left and right (stereo) disparity maps.

https://github.com/opencv/opencv_contrib/blob/master/modules/ximgproc/samples/disparity_filtering.cpp

[dorodnic]

Sorry for delayed response. Structured light sensors (the SR300) rely on SnR of pattern detection in the image, thus are usually capable of producing a confidence map. Streo is relying on matching between pixels. There either is a match or there isn't, the best pixel is always matched with "100% confidence". The only related parameter for D400 I can think of the Second Peak Threshold under Advanced Mode. This parameter will disqualify depth pixel, if the "second best" match is too close to the best match. Practically, however, this does not function exactly like a confidence threshold.

[chetansarmak]

Thank you Dorodnic So I cannot use any of rs2_options related to confidence like setting the confidence_threshold . Even I am getting same error as mentioned above while using RS2_OPTION_MOTION_RANGE. So can't I set that parameter? I used RS2_RS400_VISUAL_PRESET & RS2_OPTION_CONFIDENCE_THRESHOLD , for using both it threw error again

terminate called after throwing an instance of 'rs2::invalid_value_error' what(): object doesn't support option

Can you help me on this @dorodnic

[chetansarmak]

Thank you so much Marty Will get back to you ,If I face any hurdles while doing that.

[dorodnic]

Yes, unfortunately, Motion vs Range trade-off is also Structured Light concept - if you let more light in, you will see objects that are further away, but motions will look blurry. For stereo, changing camera exposure does just that - changes the exposure. Unless you hit over or under saturation, it should have little effect on the stereo matching. The best way to explore the various settings of the camera is via the RealSense Viewer app. It will show all the options that are supported by the device with valid ranges, and also display Advanced Mode parameters. Not all options are supported by all sensors, hence we provide supports method to check what is available at runtime. We also published a white-paper on tuning the D400 for best depth: BKMs for Tuning RealSense D4xx Cameras This might help you get more out of the device.

[chetansarmak]

Thank you so much @dorodnic . The info is very much useful for our work. In the white-paper under the Move as close as possible technique ,Can I get a graph for D435 module as well ? And how will newMaxz varies with change in disparity shift for D435?

[dorodnic]

These are excellent questions. Just to clarify, I am not the author of the white-paper :) Perhaps someone from the official support team could help find some of this information

[MartyG-RealSense]

Whilst I do not have an official chart for the D435 or a Disparity Shift formula (and are not an author of the paper either), it is possible to get a rough idea of the effects of the Disparity Shift on your particular camera model using the RealSense Viewer.

If you expand open the Viewer's 'Advanced' options section and then expand open the 'Depth Table' sub-option in that section, there is a slider to change the Disparity Shift. You can do this whilst the stream is active and see the changes in real-time.

In the demonstration below with a D415, I start with a Disparity Shift' value of '0' and then change it to '50', like in the white paper. You can observe the background of the image breaking up as the maximum distance (MaxZ) of the camera reduces as the Disparity Shift increases. If the slider is increased further then the image breaks up completely as the maximum observable distance from the camera becomes very small.

[chetansarmak]

@MartyG-RealSense Thank you so much for that method . I had done the same thing and made a chart for newMin and newmax based on the disparity shift .