search

IntelRealSense / librealsense

Intel® RealSense™ SDK
https://www.intelrealsense.com/
Apache License 2.0
7.61k stars 4.83k forks source link

Layer stuctured of PCD of scan using Intel D415 #12832

Closed JDchen-Jaden closed 7 months ago

JDchen-Jaden commented 7 months ago
Required Info
Camera Model { R200 / F200 / SR300 / ZR300 / D400 }
Firmware Version (Open RealSense Viewer --> Click info)
Operating System & Version {Win (8.1/10) / Linux (Ubuntu 14/16/17) / MacOS
Kernel Version (Linux Only) (e.g. 4.14.13)
Platform PC/Raspberry Pi/ NVIDIA Jetson / etc..
SDK Version { legacy / 2.<?>.<?> }
Language {C/C#/labview/nodejs/opencv/pcl/python/unity }
Segment {Robot/Smartphone/VR/AR/others }

Issue Description

<Describe your issue / question / feature request / etc..> THe scan distance is about 50cm, and the resultant PCD has layered looking. When evaluate the depth data for a single point. It seems the depth resolution is about 1mm. Is this normal for D415? Is this the hardware limitation? Thanks. image

JDchen-Jaden commented 7 months ago

I reset the camera using "Hardware Reset" in RealSense Viewer. The default D415 preset was used.

JDchen-Jaden commented 7 months ago

Only the threshold filter was used. 

threshold = rs.threshold_filter(min_dist=0.45, max_dist=0.53) 
depth_frame = threshold.process(depth_frame)
MartyG-RealSense commented 7 months ago

Hi @JDchen-Jaden When you say PCD, do you mean Pointcloud please? And you used the RealSense Viewer to generate a pointcloud at 50 cm distance from an object and applied a threshold filter that only shows depth values between 45 cm and 50 cm?

And when you refer to 1mm, do you mean the depth scale? If you do then yes, the default depth scale of all RealSense 400 Series cameras except the D405 model is 1 millimeter. The scale can be changed in the RealSense Viewer's 'Depth Units' options in the Stereo Module > Controls section of the Viewer's options side-panel. By default the depth unit is set to '0.001' (millimeter scale). Changing it to '0.01' would change the depth to centimeter scale. You can also set the scale to '0.0001', which can improve the depth image when depth-sensing at close range.

JDchen-Jaden commented 7 months ago

Hi @MartyG-RealSense Thanks for your quick response!

Yes, the PCD refers to the point cloud. The threshold filter is used to exclude other objects being scanned.

If D415 is designed for a depth scale of 1mm, when I change it to 0.001 mm, it will make the depth frame noisy, correct? What would be the solution to increase the depth scale resolution while still maintaining similar accuracy?

MartyG-RealSense commented 7 months ago

0.001 is another way of saying 1 millimeter, as 1 mm is equal to 0.001 meters. RealSense depth values are expressed in meters.

At 0.5 meters the D415 camera is right on the edge of its minimum depth sensing distance and it may be having difficulty rendering distance between 0.45 and 0.50 m, as depth quality starts to break up when going below a particular RealSense model's minimum distance. I would therefore recommend reducing the camera's minimum distance to pick up that 0.45-0.50 range detail more clearly by setting the Disparity Shift option to a value of '200' instead of its default '0'.

You can find Disparity Shift in the Viewer under Stereo Module > Advanced Controls > Depth Table.

I would also recommend using a depth scale of 0.0001 to improve the depth image at close-range distances from a surface like the 0.5 m you are targeting.

JDchen-Jaden commented 7 months ago

@MartyG-RealSense Thanks. I did mean the 0.001 mm depth scale not 0.001 m. Maybe 0.001mm is too aggressive. But I need the PCD depth scale at least with 0.01mm

Now I set the depth Units as 0.00001 m (0.01mm), and Disparity Shift to be 200. the depth frame was not able to show the object, a small ball. image image

Could you please elaborate more about the Disparity Shift? Is there any equation or best-practice to follow to set this parameter?

JDchen-Jaden commented 7 months ago

It seems 50 is a good value for Disparity Shift in my case. It would be helpful if you could share the methods/tumb of rules to tune the parameters.

However, the boundary is distorted. Not sure if the other surface is accurate or not. I also observed the distortion of object boundary for D405 under even and bright lighting condition and proper scan distance (same for D415). Is there any way to ensure the edge can be smoothly captured? image

MartyG-RealSense commented 7 months ago

0.0001 (not the 4-zero 0.00001) is about as low as the depth units can be set before you lose depth detail.

There is only one rule of the Disparity Shift setting. The higher the value, the more that the minimum distance of the camera is reduced, but simultaneously the more that the maximum depth detection distance of the camera is reduced. So with a high Disparity Shift value, you can end up with very little detail rendered on the depth image other than the detail that is nearest to the camera. As you found, trial and error experimentation is the best way to find which Disparity Shift value works best in a particular situation.

You could try changing the Preset option near the top of the Viewer side-panel (which is set to Custom by default) to the Medium Density preset, which provides a good balance between accuracy and the amount of detail on the depth image. The Default preset, meanwhile, can provide clean edges and reduced pointcloud 'spray'.