RealSense-Customer-Engineering
commented
5 years ago [Realsense Customer Engineering Team Comment] The RealSense development team will look into this issue.
Closed btravers1 closed 7 months ago
RealSense-Customer-Engineering
commented
5 years ago [Realsense Customer Engineering Team Comment] The RealSense development team will look into this issue.
btravers1
commented
5 years ago On further investigation, I found that autoexposure really doesn't work well when using the USB2 interface, regardless of the resolution or framerate. It behaves much better on USB3. This observation is independent of the host platform or software. Certain USB2 modes are better than others. For example, 640x480 @ 30FPS behaves much better than 480x270 @ 15FPS. Either one will get locked into weird modes, though. The most common "mode" I've observed is triggered in full sunlight with a scene containing reflective surfaces. In this scenario, autoexposure will lock up in an overexposed condition. The only way to reset it is to either reset the camera or globally reduce scene lighting for a few frames. Another mode seems to be triggered by a long skinny reflector on a less reflective surface, sweeping horizontally across the scene. In this mode, exposure will continually oscillate severely. (e.g. alternating frames will be drastically overexposed or underexposed) Again, the only way to get the camera out of this "mode" is to reset the camera or globally reduce scene lighting for a few frames.
Anyway, that's what I've observed - if you need any more details or examples, please let me know.
btravers1
commented
5 years ago I forgot to mention: I added some IR filters and it made no difference in the behavior, so it doesn't appear to be an issue with the sensor.
btravers1
commented
5 years ago Important Update:
I did a bit more playing and found that some of the behavior can be easily observed using the RealSense Viewer utility. I discovered that the issue likely isn't related to the autoexposure algorithm but rather the exposure control on the image sensor / ASIC, but only when using USB2.
If you connect a camera on USB 2 and disable autoexposure, setting the exposure time to anything below 61 causes the image to become overexposed. (almost like a scaling / wrap-around error) Consequently, I assume the autoexposure algorithm is working correctly, but it's getting "locked up" in the following scenario:
This doesn't completely explain all the behavior, but I believe fixing this bug would fix the issues we've been experiencing in outdoor environments.
Obviously, when this occurs, the disparity map becomes almost unusable. (full of holes) However, I am impressed at how well it works, given the image quality in this condition. Kudos!
RealSense-Customer-Engineering
commented
5 years ago [Realsense Customer Engineering Team Comment] Hi,
Is there any platform limitation for you so that you have to USB2.0?
We reproduced this problem. But USB2.0 is not our main POR support. The engineering team will work on it with low priority.
Sorry for the inconvenience.
Thanks,
btravers1
commented
5 years ago Thank you for your response. Our hardware platform is centered around an i.MX6, so we're pretty much stuck with USB2. We could go to a different processor with USB3, but there is generally a significant price increase as you have to enter a high-performance processor family. (higher silicon costs, faster RAM, moving to active thermal design, higher tolerances, etc.) Obviously, that's changing, but we're not there yet :-)
Just as a side comment, my impression of the D400 series camera is that it's targeted towards low-cost, embedded consumer-grade applications. If that were the case, it would make sense to me to primarily target USB2 and provide USB3 for the people who need high resolution or framerates. In my thinking, if you target a high-end processor, you run the risk of a future "cost reduction" projects dropping the D400 platform altogether. (if you have the processing power, why not move disparity map calculations inside the application processor) I'm sure the product line managers have more information and made a decision that made sense, but that's my two cents if you're looking for feedback.
Thank you, Ben
Issue Description
When a D420, D430, or D435 camera connected via USB 2.0 is exposed to a very bight light that saturates the imager (i.e. direct sunlight or a bright flashlight), autoexposure jumps to a maximum "gain" and then will not recover.
At that point, exposing the camera to complete darkness (i.e. covering the lenses) causes the camera to lock up for a few seconds. Once it comes back, there seems to be a data alignment issue in the images. If using the RealSense Viewer, you'll see a few "MIPI errors." Once it comes back, the depth and IR images are wonky. (technical term)
Conditions
This issue only occurs when using the camera on a USB 2 bus at modes >= 480x270 @ 15fps. At 480x270 @ 6fps, I don't get the MIPI errors, but the autoexposure still "locks up" until the camera is exposed to complete darkness. Considering the fact that the camera can stream much higher data rates on the USB 2 bus (e.g. 640x480 @ 30fps), I don't believe this has anything to do with data throughput. The problem is independent of the host platform.
Likely Cause
In my opinion, it is very likely this problem is caused by a "wrap-around" bug. When the imager is saturated, the autoexposure algorithm is likely decrementing something like "exposure" or "gain" too far, causing a wrap-around. (i.e. decrementing a byte with a value of 0 will change the value to 255)
Once the camera is exposed to complete darkness, the autoexposure is trying to increment the "gain." Since the gain is likely already above it's maximum allowed value, something is dying, and a reboot occurs. (watchdog, segfault, etc.)