IntelRealSense / librealsense

Intel® RealSense™ SDK
https://www.intelrealsense.com/
Apache License 2.0
7.53k stars 4.81k forks source link

Is the Realsense D435i depth acquisition principle based on structured light ranging? #10409

Closed hanbeihero closed 2 years ago

hanbeihero commented 2 years ago

Required Info
Camera Model { R200 / F200 / SR300 / ZR300 / D400 }
Firmware Version (Open RealSense Viewer --> Click info)
Operating System & Version {Win (8.1/10) / Linux (Ubuntu 14/16/17) / MacOS
Kernel Version (Linux Only) (e.g. 4.14.13)
Platform PC/Raspberry Pi/ NVIDIA Jetson / etc..
SDK Version { legacy / 2.<?>.<?> }
Language {C/C#/labview/nodejs/opencv/pcl/python/unity }
Segment {Robot/Smartphone/VR/AR/others }

Issue Description

<Describe your issue / question / feature request / etc..>

MartyG-RealSense commented 2 years ago

Hi @hanbeihero The RealSense 400 Series depth cameras (such as the D435i model) use Stereo Depth technology, where a depth image is constructed from left and right infrared images.

The now-retired RealSense SR300 and SR305 models use coded light technology, which is similar to structured light.

hanbeihero commented 2 years ago

Does Realsense D435i depth ranging belong to structured light ranging in principle? Can the Realsense D435i produce good depth images in outdoor lighting conditions? What is the difference between Stereo Depth and structured light? These issues seem to be very controversial on the Internet.

MartyG-RealSense commented 2 years ago

No, stereo depth technology is not related to structured / coded light based depth technology.

Yes, 400 Series stereo depth cameras work excellently outdoors. The SR300 / SR305 coded light cameras are suited to indoor use and have a short maximum depth sensing range of around 1.5 meters, whilst most 400 Series depth cameras (except the D405 model) can depth-sense to a farther distance.

I will provide the official depth sensing descriptions for stereo and coded light from Intel's data sheet documents.

Stereo depth The Intel RealSense D400 series depth camera uses stereo vision to calculate depth. The stereo vision implementation consists of a left imager, right imager, and an optional infrared projector. The infrared projector projects a non-visible static IR pattern to improve depth accuracy in scenes with low texture. The left and right imagers capture the scene and send imager data to the depth imaging (vision) processor, which calculates depth values for each pixel in the image by correlating points on the left image to the right image and via the shift between a point on the Left image and the Right image. The depth pixel values are processed to generate a depth frame. Subsequent depth frames create a depth video stream.

Coded light The Intel RealSense Depth Camera SR300 series depth camera uses multiple components to generate a depth image. To generate a depth frame, the IR projector illuminates the scene with a set of predefined, increasing spatial frequency coded IR vertical bar patterns. These patterns are warped by the scene, reflected back and captured by the IR camera. The IR camera pixel values are then processed by the vision processor to generate a depth frame. Subsequent depth frames create a video stream that is transmitted to the host system.

hanbeihero commented 2 years ago

Excuse me, is non-visible static IR easily overwhelmed by outdoor light?

MartyG-RealSense commented 2 years ago

If there are visible light sources in the observed scene that include IR or near-IR frequencies then most 400 Series camera models (including D435i) can see them.

The D405 model cannot as its sensors are equipped with an IR Cut filter that make IR frequencies invisible to the camera.

hanbeihero commented 2 years ago

Thank you, your reply helped me a lot!

hanbeihero commented 2 years ago

When the outdoor light is too strong to drown the infrared light spot of the D435i camera, will the camera obtain depth information with passive Binocular Stereo Vision?

MartyG-RealSense commented 2 years ago

If a scene is strongly lit by sunlight then RealSense 400 Series cameras can use the ambient light in the scene to analyze objects and surfaces for depth detail instead of analyzing the infrared pattern projection (whose visibility would be reduced by a brightly lit scene). So in a scene with strong sunlight the camera should obtain depth information from an object / surface one way or another, whether from analyzing the dots projected onto objects or by the natural sunlight cast onto objects.

hanbeihero commented 2 years ago

Does this have a manual to introduce the working mode of D435i in various environments?

MartyG-RealSense commented 2 years ago

The best match to your request may be Intel's Depth Map Improvements for Stereo-based Depth Cameras on Drones guide for using RealSense with outdoor drones.

https://dev.intelrealsense.com/docs/depth-map-improvements-for-stereo-based-depth-cameras-on-drones

MartyG-RealSense commented 2 years ago

Hi @hanbeihero Do you require further assistance with this case, please? Thanks!

hanbeihero commented 2 years ago

Everything is going well so far, but I have some doubts about the imaging principle of the Realsense D435i. According to the classification on the Internet, the imaging principle of the Realsense D435i is similar to that of structured light, and I am a little worried about the outdoor effect of the camera.

MartyG-RealSense commented 2 years ago

The stereo depth technology of the 400 Series cameras is different to structured light principles. In the link below, Intel provide a guide to the principles of stereo depth cameras such as the 400 Series.

https://www.intelrealsense.com/stereo-depth-vision-basics/

Under the heading Structured-light approach in that guide, it explains how structured light differs from stereo depth.

400 Series cameras are fully capable of being used outdoors and can actually perform better in sunlight, as described in the section of Intel's camera tuning guide linked to below.

https://dev.intelrealsense.com/docs/tuning-depth-cameras-for-best-performance#section-use-sunlight-but-avoid-glare

MartyG-RealSense commented 2 years ago

Hi @hanbeihero Do you require further assistance with this case, please? Thanks!

MartyG-RealSense commented 2 years ago

Case closed due to no further comments received.