Closed LadaOndris closed 2 years ago
MartyG-RealSense
commented
2 years ago Hi @LadaOndris Could you test whether you see a noticable improvement if you enable the Emitter Always On option in the Viewer, please?
The D435's projector pulses in line with exposure, whilst the projector of the D415 model is always on. Enabling 'Emitter Always On' replicates the behaviour of the D415's always-on projector.
LadaOndris
commented
2 years ago @MartyG-RealSense Thank you, Marty, for such a quick response. Unfortunately, I cannot find this option in the menu. See the image below.
MartyG-RealSense
commented
2 years ago It should be located just below the camera name.
LadaOndris
commented
2 years ago Do you mean in the More selection menu? I don't see it there.
MartyG-RealSense
commented
2 years ago My apologies, I was thinking of 'Hardware Reset' when I meant 'Emitter Always On', which is elsewhere in the side-panel.
It is not clear why this option is not being listed for your D435. Do you have both of the Infrared options listed, or are one or both of the Infrared streams missing?
LadaOndris
commented
2 years ago Yes, I can see both of them and they both work. Shouldn't I also be able to see the Change ROI button? I'm not sure, however, whether this model supports it.
MartyG-RealSense
commented
2 years ago The Set ROI button should appear next to 'Enable Auto Exposure' once the Stereo Module is enabled.
LadaOndris
commented
2 years ago Exactly, that's what I mean. As you can see in the images I posted above, there is no such button.
MartyG-RealSense
commented
2 years ago Which version of the RealSense SDK are you using, please?
LadaOndris
commented
2 years ago You are correct. I indeed had an old SDK version. To answer your previous questions, I can notice only a slight improvement when "Emitter Always On" is enabled. Setting ROI helps significantly in the ability of the camera to set the exposure more reliably.
Nonetheless, the image quality in sunlight is diminished. Now I consider only the case when the camera lens is directed away from the Sun and only the object is sunlit. See the image below I just captured. It seems the exposure is correct, which is visible from the infrared image. Any ideas what causes the decreased quality and how to improve it?
MartyG-RealSense
commented
2 years ago The black areas on the depth image seem to correspond to sections on the infrared image that are black or dark grey. This includes the black areas of darkness inbetween the fingers and the shadowy dark grey area around the shoulder and the lower side of the hand.
The infrared and depth images strongly resemble images that are produced when the camera's projector is off or the Laser Power option's value is minimized / zero. In your earlier Viewer image at https://github.com/IntelRealSense/librealsense/issues/10309#issuecomment-1065281599 though, you had the projector enabled and Laser Power set at the default of '150'.
Have you tried maximizing the Laser Power value to
As you are using an auto-exposure ROI, you could also try adjusting the auto-exposure Mean Intensity Setpoint value, which can be found under Advanced Controls > AE Control in the Viewer's options side-panel. An auto-exposure ROI tries to maintain the intensity of the pixels in the ROI-highlighted area at an average defined by the Setpoint vlaue.
See point 3 of the section of Intel's camera tuning guide linked to below for more details about ROI.
MartyG-RealSense
commented
2 years ago Hi @LadaOndris Do you require further assistance with this case, please? Thanks!
LadaOndris
commented
2 years ago I appreciate your help. I still need to try the setting in direct sunlight, hopefully in a couple of days. I'll let you know then.
MartyG-RealSense
commented
2 years ago Okay, thanks very much for the update. Good luck!
LadaOndris
commented
2 years ago I tried to increase the laser power as you mentioned. It does not help when there is already enough natural light, which is the case that I discuss here. For demonstration, both images below had laser power at maximum.
You said in your comment above that the black areas correspond to black or dark gray places due to underexposure. According to my observations, it is true only partly. Notice the bottom right part of the image in my previous post---the forearm is almost black in infrared but the camera still produced valid depth.
The black areas increase with the decreasing distance from the sensor. See the below images. The first one depicts the hand at a distance of 30 cm, while the second one at 16 cm. It is clearly visible that with shorter distances, the number of black areas increases.
You suggested changing the Mean Intensity Setpoint, which really comes useful when adjusting the intensity. It does not, however, help me get rid of the black areas in this case. Changing in whichever direction, it causes either underexposure or overexposure of certain areas. There is no value that solves this problem.
I should mention that the above images were captured using the Hand preset. If I set the preset to Default or High density, then the number of black areas significantly decreases, which leads me to believe that the black areas are created by thresholding and other settings that cause the camera to discard those depth values. See the image below, which was taken using the Default preset:
So I am wondering what settings may cause these black areas to appear in full sunlight.
MartyG-RealSense
commented
2 years ago Parts of a black or dark grey area in a scene can still return depth information if light is being cast onto those parts, as demonstrated with a black office chair at https://github.com/IntelRealSense/librealsense/issues/6757
When depth-sensing arms, it is a known phenomenon that the wideness of the black areas can increase as the arm / hand becomes closer to the camera.
Reducing the Depth Units under the 'Controls' section of the RealSense Viewer's Stereo Module options from its default value of '0.001' to '0.0001' can enable a better depth image at close range.
MartyG-RealSense
commented
2 years ago Hi @LadaOndris Do you require further assistance with this case, please? Thanks!
LadaOndris
commented
2 years ago Sorry for the late answer. I have been occupied with other work. I will try changing the depth units and I will let you know. Thank you for your answers so far!
MartyG-RealSense
commented
2 years ago Okay, no problem. Thanks very much for the update and I look forward to your next report. Good luck!
MartyG-RealSense
commented
2 years ago Hi @LadaOndris Do you have an update about this case that you can provide, please? Thanks!
MartyG-RealSense
commented
2 years ago Hi @LadaOndris Do you require further assistance with this case, please? Thanks!
MartyG-RealSense
commented
2 years ago Case closed due to no further comments received.
Issue Description
Hello, I am writing regarding an issue we have with a D435 camera. The problem is that the camera is supposed to be used in various conditions with different lighting. Moreover, the lighting constantly changes during the day, depending on the sunlight intensity. I seem to fail to find a solution to maintaining solid depth image quality. I studied a few RealSense articles with explanations and recommendations, so I am familiar with the concept of exposure, which is the problem we are having. The image is either underexposed or overexposed. I read the following article for instance: https://www.intel.com/content/dam/support/us/en/documents/emerging-technologies/intel-realsense-technology/BKMs_Tuning_RealSense_D4xx_Cam.pdf
When an object is exposed to direct sunlight, I found the best exposure to be set at 3 μs. However, in the evening when only a room light is shining while using the camera's laser at 100 mW, the best exposure seems to be around 4 000 μs, which is a huge difference from the previous example. Both cases consider an object positioned very close to the camera. If the object moves away, it may become underexposed.
Our goal is to use the camera for gesture recognition and determining the fingers' positions of a moving hand in the direction of the camera, approaching it from 1 meter to 10-15 cm, which requires different exposure times to avoid underexposure or overexposure. Thus, it is unrealistic to use manual exposure. To our dismay, the autoexposure is too slow and incorrect. I found that it requires the hand to stop at a certain position and wait a little while for the camera to adjust. In some cases, it even fails to adjust sufficiently, and, therefore, the image quality is much lower in comparison to manual exposure.
From the article I mentioned above, I understand that options such as ROI and Mean Intensity Set Point can be used with autoexposure. I do not understand how to use the Mean Intensity Set Point. I also do not see the ROI button in Intel RealSense Viewer.
I need to find settings that will not require adjustments in the final product depending on the time of the day and other conditions. The solution must give reliable images in good quality. Are there solutions to this?
The following images demonstrate what I described above:
Week sunlight, autoexposure. Really poor image quality.
Much stronger sunglight. I experimented with both autoexposure and manual exposure, but none provided me with good results.
Sometimes, I am able to get pretty decent images.
