IntelRealSense / librealsense

Intel® RealSense™ SDK
https://www.intelrealsense.com/
Apache License 2.0
7.64k stars 4.83k forks source link

On the data disorder of d455 depth image #7777

Closed weiyi-jie closed 3 years ago

weiyi-jie commented 4 years ago

In my normal use process, the camera d455 has the phenomenon of depth data dislocation. When I restart the device, it has a certain probability to repair and output complete and correct depth data. (this phenomenon is completely random).

My version number: librealsense2.so.2.39.0

I would like to ask you what causes this phenomenon and how to avoid it. Thank you very much!

MartyG-RealSense commented 4 years ago

Hi @weiyi-jie Could you provide more information please about what you mean by 'depth data dislocation'. Does the depth data freeze or disappear? And are you using the camera indoors or outdoors? Thank you very much.

weiyi-jie commented 4 years ago

Hi @weiyi-jie Could you provide more information please about what you mean by 'depth data dislocation'. Does the depth data freeze or disappear? And are you using the camera indoors or outdoors? Thank you very much.

Hello, @MartyG-RealSense I use it indoors. The depth image data has not been frozen, and the real-time depth data is still being transmitted. Only when the image is displayed, there is a long transverse crack, the whole image is split and moved downward, or the image acquisition is incomplete, only part of the image can be obtained, and the screen flickers.

MartyG-RealSense commented 4 years ago

Hi @weiyi-jie I tried to visualise what you are experiencing. It would be very helpful if you could provide an image in this discussion though if that is possible.

Does the flickering depth image that you are experiencing look anything like the one in the link below please:

https://github.com/IntelRealSense/librealsense/issues/7268

weiyi-jie commented 4 years ago

HI @MartyG-RealSense, I'm sorry, it was my negligence that I forgot to upload the problem image. My problem image is shown in the figure below. There will be all these phenomena 0 1 2 0 1 2

Here is the normal image 0 0

The camera distance is 3.9m

MartyG-RealSense commented 4 years ago

Are you using the RealSense Viewer program to generate images, please? If you are using the Viewer, could you check please whether you have a filter enabled in the Post-Processing controls that is called Threshold Filter. If this filter is enabled - shown by a blue (On) icon beside the Threshold Filter name - then by default it limits the D455 camera's observable depth range to only 4 meters.

You can either left-click on the blue icon to change its color to red (Off) or use the Maximum Distance slider of this filter to set it to its maximum distance.

image

image

weiyi-jie commented 4 years ago

Sorry, @ MartyG-RealSense I didn't use realsense viewer. I used demo program in librealsense master / build / examples. I can't find this problem in realsense viewer for the time being (probably due to too few tests).

I will test as you describe.

MartyG-RealSense commented 4 years ago

Please do update me when you have done tests. Thanks!

weiyi-jie commented 4 years ago

@ MartyG-RealSense Hello, I set the settings according to what you said. I continued to test without any repair, but I used the realsense viewer without any error. Is it true that the depth data acquisition methods of realsense viewer are different? How was it acquired? Can I get its data acquisition program?

MartyG-RealSense commented 4 years ago

The RealSense Viewer is open-source code that you can modify for your own needs.

https://github.com/IntelRealSense/librealsense/tree/master/tools/realsense-viewer

You may certainly experience differences in the images that you get from a program that you create yourself, compared to the images achieved in the RealSense SDK. This is partly because the RealSense Viewer enables a lot of functions such as post-processing and colorization by default, whereas these functions have to be manually programmed into your application when creating your own scripts. Example programs that do not include certain features like the Viewer does may also generate different-looking images.

I ran some tests with the Viewer, pointing the camera at a set of windows as you seem to be doing in the image above. The windows were visible normally if Auto-Exposure was enabled in the Stereo Module controls of the Viewer, and the windows disappeared if Auto-Exposure was disabled. As the Viewer has auto-exposure enabled by default when it launches, I wonder if the example that you used did not have Auto-Exposure enabled.

weiyi-jie commented 4 years ago

@Martyg realsense Sorry to reply you now. I modified it as you said, and then tested it many times without any problems; but when I power down the camera and plug it in again, it will happen again. Now my camera will report the following error message:

24/11 16:35:02,671 WARNING [140128753518336] ( sensor.cpp:632 ) Exception was thrown when inspecting Raw RGB Camera property Controls white balance of color image. Setting any value will disable auto white balance

24/11 16:35:02,935 WARNING [140128921757504] ( types.cpp:78 ) hwmon command 0x7d( 0 0 0 0 ) failed (response -21= No data to return)

MartyG-RealSense commented 4 years ago

I will seek advice from Intel about your latest problem with the above errors appearing. Thanks very much for your patience!

MartyG-RealSense commented 4 years ago

I received guidance from Intel. Could you confirm please in the RealSense Viewer if the depth can stream without any issues and that the RGB cannot stream.

weiyi-jie commented 4 years ago

Yes, in realsense viewer, depth can be streamed without any problems, and RGB can be streamed normally.

MartyG-RealSense commented 4 years ago

Thanks very much. Are you still experiencing the WARNING messages or did they only happen once when plugging the camera in.

weiyi-jie commented 4 years ago

Yes, there will always be a warning message at the terminal, not only at the first startup.

MartyG-RealSense commented 4 years ago

I reviewed your case agan from the beginning for new insights and noted that there is a probability of the camera functioning correctly if the camera is reset.

I also note that you were encountering these broken images in an example demo program. If you are compiling your own programs, would it be possible please to try adding a hardware_reset() instruction to the program please? This should reset the camera automatically when the program launches, like when the Hardware Reset option is selected from the 'More' menu in the RealSense Viewer.

C++ https://github.com/IntelRealSense/librealsense/issues/2161#issuecomment-408779150

Python https://github.com/IntelRealSense/librealsense/issues/3329#issuecomment-475293475

yzkzero commented 4 years ago

Hi Weiyi-jie, Hi MartyG,

I think the problem is very similar with what I met and mentioned in: https://support.intelrealsense.com/hc/en-us/community/posts/360051428274-Incorrect-depth-image-by-D455

From your screenshot, I also noticed the "band and shifting" phenomenon in this reply: https://github.com/IntelRealSense/librealsense/issues/7777#issuecomment-729343146

So it looks like a common issue of D455. It happens very frequently and it should be solved as soon as possible.

MartyG-RealSense commented 4 years ago

@yzkzero I do not want to make a comment that pre-empts any official technical information that Intel releases for the D455. I do understand your frustration and ask for your kind patience, please.

weiyi-jie commented 4 years ago

Hi MartyG,I'm sorry to reply you now. I just came back from a business trip. I tested this method as you said at the first time. So far, there are no problems before the occurrence. Thank you very much for your help, thank you!

MartyG-RealSense commented 4 years ago

@weiyi-jie That's great to hear - thanks very much for the update!

MartyG-RealSense commented 3 years ago

Hi @weiyi-jie Do you require further assistance with this case, please? Thanks!

weiyi-jie commented 3 years ago

HI @MartyG-RealSense Thank you very much for your help. This problem does not appear at present.

weiyi-jie commented 3 years ago

Hi @MartyG-RealSense , I'd like to ask how to set the depth data to obtain a relatively complete depth image in the sun.

I use realsense_ The viewer can get relatively complete data by setting preset: default. How to set this mode in the program?

weiyi-jie commented 3 years ago

The first image is the depth 222 This is through realsense_ viewer--- preset: default 111 Is it necessary to set something in the program? This is the result. Thanks!

MartyG-RealSense commented 3 years ago

If you wish to set the SDK's Default preset with scripting, there is a method for loading existing presets supplied in the SDK that is simpler than the code that is typically involved in loading a custom-made preset.

In the example code in the link below, change RS2_RS400_VISUAL_PRESET_HIGH_DENSITY to RS2_RS400_VISUAL_PRESET_DEFAULT to load the Default preset.

https://github.com/IntelRealSense/librealsense/issues/7399#issuecomment-697609404

If you want to program in Python, the link below should be a useful reference.

https://github.com/IntelRealSense/librealsense/issues/2577#issuecomment-432137634

weiyi-jie commented 3 years ago

This is feasible. Thank you for your help @MartyG-RealSense

MartyG-RealSense commented 3 years ago

Hi @weiyi-jie Do you require further assistance with this case, please? Thanks!

weiyi-jie commented 3 years ago

@MartyG-RealSense Thank you for your help. Now it's working,Thank you.

MartyG-RealSense commented 3 years ago

Great news @weiyi-jie - thanks for the update! I will close this case now as you have achieved a solution.