D455 shows reddish color in RGB image when there is sunlight

[vijujames]

The new D455 shows red color in the image when there is sunlight. See the attached picture. This was taken around 3pm with the camera facing West. Is there a setting that needs to be changed, or, is it a hardware issue? (This does not happen with D435 and D435i)

[MartyG-RealSense]

Hi @vijujames This effect looks like what is known in photography as a 'color cast', where the RGB image is tinted incorrectly with a particular color.

Tints may be able to be removed using the color balance controls under the Controls sub-section of the Viewer's RGB Camera controls in the options side-panel.

I note though the multiple get_device_time_ms errors in the log section at the bottom of your Viewer window. This was a common problem in the 2.34.0 version of the Viewer that you are using in the image.

Also, official support for the D455 camera was not introduced until version 2.36.0. I would therefore strongly recommend updating your RealSense SDK to a newer version.

[vijujames]

Hi @MartyG-RealSense Is the SDK 2.36.0 needed only for realsense-viewer? Can D455 streams be recorded using 2.34 SDK?

[MartyG-RealSense]

When support is introduced for a camera in the SDK, it applies to the SDK in general and not just the Viewer program on its own. Having said that, the D455 is able to be recognised and correctly named as a D455 in the Viewer in some earlier versions than 2.36.0 (including yours). I just tested it myself in SDK 2.35.2.

The continuously generating errors in 2.34.0 are still a good reason to upgrade though - if you look in the bottom log window of the image that you provided, you can see that you were experiencing this error yourself.

I ran some tests in the Viewer with my D455 and found that I could recreate your image if the Hue setting in the RGB controls was increased to 100 or more. So I suggest checking in the Viewer what your Hue is set to when you get that pinky-purple tint. I found that if the Hue is changed then that value persists even if you close the Viewer and re-launch it, until you change the value again..

[vijujames]

Hi, @MartyG-RealSense We did some more testing with SDK 2.36 and D455 and D435i. It looks like changing Hue changes color, but at Hue=0, D455 still shows colorization. It is better indoor, but once you face it outdoor, colorization happens. See the following pictures. Last one is D435i. It looks to me something wrong with the unit I have. What do you think?

D455 indoor Hue=0

D455 Outdoor Hue=0

D455 Outdoor Hue=60

D435i Outdoor Hue=0

[MartyG-RealSense]

I had a discussion with my manager about this case. RealSense support are aware of the issue and are investigating it.

[agrunnet]

I would ask for some more data from your side, if possible. It looks your specific example MAY be due to a bad IR-cut filter. Do you see any of these adverse effects indoors away from IR light?

[vijujames]

@agrunnet please let me know what data you need from our side. We haven't noticed this issue indoors. It happens only if we point the camera outside where there is sunlight as in the above pictures. The picture with the trash can above looks ok, but not the ones facing outside.

[agrunnet]

@vijujames We were able to reproduce your scene and we see the same effect. No more info needed from you, as we will continue to look into this. Thanks for bringing it to our attention.

[sam598]

I have also experience this issue outdoors in direct sunlight. It seems to happen above a certain amount of light or camera exposure level, and always comes out from the lower right corner of the camera sensor. Maybe the sensor is becoming over saturated?

@vijujames a temporary workaround might be to place a neutral density filter in front of the color sensor. This will reduce the amount of light hitting the sensor without significantly affecting color. The filters do come in square sheets that can be cut to size.

[agrunnet]

@sam598 and @vijujames (updated) Yes, I can confirm that is expected behavior for extremely bright scenes and is a saturation issue. It happens for extremely bright scenes when the sensor is set to very short exposure times for the sensor. This RGB sensor on the D455 is global shutter and has a very good low light sensitivity to reduce blurring during fast motion indoors under most normal lighting conditions.

For these extreme cases, I refer to our white paper on Optical Filters, and recommend using ND filter (aka. sun glasses). A polarizer can also help, especially as it reduces significantly the strong light reflecting off of surfaces.

[sbargon]

@agrunnet This happens even on a cloudy day, how do you suggest an ND filter is fitted?

[agrunnet]

Yes. ND 1.5 - 2.0 should work fine. This is a saturation artifact of a very sensitive global shutter sensor. The pixels are quite large (4.5x bigger in area than D435 color sensor) and well suited for fast motion in normal outdoor light and all indoor operations. Under very bright conditions the sensor starts to saturate. Instead of us limiting the exposure and over-saturating the image, we are permitting the brighter conditions in our FW, so as to extend the range, but the warning that you are above the exposure limits is the onset of the purple from the lower right corner.

[sam598]

@sbargon it is possible to purchase ND filters as square sheets, which you can then cut to size. For example: https://www.bhphotovideo.com/c/product/38380-REG/Norman_810551_810551_Neutral_Density_Pack.html

In terms of mounting them to the camera, the easiest solution is tape but that is far from ideal. If you have access to a 3D printer someone could design a filter holder that wrapped around from the back of the camera to the front. This would be a less destructive path.

@agrunnet if the sensor is very sensitive is there any concern about sunlight permanently damaging the pixels? Obviously any extremely bright or magnified light source can cause damage, but in "every day outdoor use" is there any concern? Also since they use the same sensor, what about the IR cameras?

[agrunnet]

@sam598 thanks for the great link. And to answer your question, no there is no permanent damage at all. And yes your observation is absolutely correct about the stereo RGB sensors. They are the same as the RGB imager and will also experience the same saturation issue. However because the effect is almost identical for both left and right imagers there will be much less of an impact to the depth, which is one reason to allow this gradual saturation and coloration to occur.

[FrankCreen]

The new D455 shows red color in the image when there is sunlight. See the attached picture. This was taken around 3pm with the camera facing West. Is there a setting that needs to be changed, or, is it a hardware issue? (This does not happen with D435 and D435i)

Hi ！I test the code below with a 75FPS, could you show your result? Thanks very much!

Test Program

// include the librealsense C++ header file
#include <librealsense2/rs.hpp>

#include <opencv2/opencv.hpp>

#include <chrono>

using namespace std;
using namespace cv;

int main()
try
{
    //Contruct a pipeline which abstracts the device
    rs2::pipeline pipe;

    //Create a configuration for configuring the pipeline with a non default profile
    rs2::config cfg;
    //Add desired streams to configuration
    cfg.enable_stream(RS2_STREAM_COLOR, 480, 270, RS2_FORMAT_BGR8, 90);

    //Instruct pipeline to start streaming with the requested configuration
    pipe.start(cfg);

    //Sensor Settings
    auto sensor1 = pipe.get_active_profile().get_device().query_sensors()[1];
    sensor1.set_option(RS2_OPTION_ENABLE_AUTO_EXPOSURE, 1);
    auto sensor2 = pipe.get_active_profile().get_device().query_sensors()[1];
    sensor2.set_option(RS2_OPTION_AUTO_EXPOSURE_PRIORITY, 0);

    // Display in a GUI
    const auto window_name = "Display Video";
    namedWindow(window_name, WINDOW_AUTOSIZE);

    //用于计时
    //show time
    auto start = chrono::system_clock::now();
    int frame = 0;

    while (waitKey(1) < 0 && getWindowProperty(window_name, WND_PROP_AUTOSIZE) >= 0)
    {
        rs2::frameset data = pipe.wait_for_frames(); // Wait for next set of frames from the camera

        //获取彩色图像帧
        rs2::frame color = data.get_color_frame();

        //保存彩色图像
        Mat image(Size(480, 270), CV_8UC3, (void *)color.get_data(), Mat::AUTO_STEP);

        // Update the window with new data
        imshow(window_name, image);

        frame++;
        //one loop end
        auto end = chrono::system_clock::now();
        auto duration = chrono::duration_cast<chrono::microseconds>(end - start);
        double timecost = double(duration.count()) * chrono::microseconds::period::num / chrono::microseconds::period::den;

        cout << frame / timecost << " FPS" << endl;
    }

    return 0;
}
catch (Exception ex)
{
    cout << "Error!!!" << endl;
    cerr << ex.msg << endl;
}

[MartyG-RealSense]

Hi @FrankCreen Please post your question as a new issue using the New Issue button at the link below and do not post it on multiple D455 cases if the question is unrelated to that case. Thank you.

https://github.com/IntelRealSense/librealsense/issues/

[dingyu20]

Hi, is there any updates on this issue? @MartyG-RealSense

[hanmekim]

+1 @dingyu20's comment above.

As I wrote here, I also have the same issue, and I have been waiting for a more supportive follow-up from Intel RealSense (@MartyG-RealSense) while using it without color, but nothing heard since then.

Based on my experience (also @sbargon's post above), it's not an issue only under bright sunlight, but is rather constantly happening under general outdoor scenarios making it unusable outdoor at all.

Except this issue, I like D455 which is well integrated and built for robotic applications, so I really hope that Intel RealSense team now comes back with more reasonable supports soon rather than referring to the CTO's post and a potential solution with a ND filter. E.g.:

• any upcoming firmware/hardware update
• provide a ND filter and a bracket (at least describing where to buy it and how to attach to D455 if that's something acceptable by customers)

[MartyG-RealSense]

@dingyu20 @hanmekim I will refer your comments to Intel. Thanks very much for your ongoing patience,

[RealSenseSupport]

There is a document detailed about this issue, which is located on the FW page: https://dev.intelrealsense.com/docs/firmware-releases

Documentation Specification Clarification - October Release (link to PDF attached) Refer to the Intel RealSense product documentation (PDF) included in the archive for the latest device and documentation errata, specification clarifications and changes.

[CSTEZCAN]

Hi!

I have the same problem and let me elaborate on this issue.

I'm founder of a multispectral camera company and we had this exact issue with some filter producers back in 2017. Their filters advertised as "stabldge" (you can search the name of the company) but in reality their products weren't. I'm 100% sure Intel using the same filters to block out the NIR because it has the same problem. The NIR light bleeds through horizontal-vertical edge of the filter and causes this problem.

http://midopt.com/wp-content/uploads/2016/04/MidOpt_StablEDGE-Theory-Curves_2_web.jpg

This is a production issue and intel must recall and replace all of the affected units like we had to.

Intel; You can't solve this issue with firmware update.

I'm returning my D455 and I'll advise my customers NOT to buy them.

Regards

[hanmekim]

FYI, I tried a ND filter as shown below. Beside extra $50.45 (a ND filter inc. shipping) and its ugly looking, I found that can't be a solution at least in my case (a moving platform). Basically, it increases the exposure time eliminating the color issue, but consequently it becomes easier to get motion blur. I bought a D435i instead to give it a try, but not sure yet because of its rolling shutter (RGB). Definitely, I wouldn't recommend D455 for outdoors.

[FPSychotic]

Why is this closed? Returning the mine, it is not only the hue tint, the light adaption is absolutely slow. Unusable. No for outdoor, not for mobile applications, not for AI, no for computer vision maybe for webcam or DARPA challenge. https://www.facebook.com/groups/RealSenseExperts/permalink/2292731394193698/

I added a video in the Realsense group. Why intel is not doing a recall? Sun glasses, seriously?

[FPSychotic]

That light changes adaption....ahh and there is greenish tint in the left side, it's not only reddish. I miss yellow tint, it is my favourite color. https://youtu.be/LzL1Vo-Bow8

[WingRS]

We happen to have the same issue with out D455s, in desert environment, (the drawing are done over the human, it doesn't change the color or anything in the image)

[MartyG-RealSense]

Hi @WingRS For information about this D455 phenomenon, please download Intel's Specification Clarification PDF document using the link below and read the D455 RGB Camera In Bright Sunlight section on the final page of the document.

https://www.intelrealsense.com/download/13629/

[marianbrunet]

Has anyone tried to remove the lens in from of the RGB camera to see what happens? D415 and D435 don't have this issue and they have no lenses in front of the cameras. Please let me know your thoughts.

[MartyG-RealSense]

Hi @nievesbrunet The lenses of RealSense camera sensors are not designed to be removed and doing so risks breaking the sensor. The OmniVision OV9782 'global shutter' RGB sensor on the D455 is also a different component to the OmniVision OV2740 'rolling shutter' one on the D415, D435 and D435i models.

[marianbrunet]

Thank you @MartyG-RealSense. Any other advice? I'm using an ND4 filter, and can still see some pinkish color on the right side of the image. Thank you.

[FPSychotic]

Because they are faulty and should be refunded and recalled. Their filter documents is just a absurd tale.

On Tue, 8 Mar 2022, 20:21 Maria Nieves Brunet, @.***> wrote:

Thank you @MartyG-RealSense https://github.com/MartyG-RealSense. Any other advice? I'm using an ND4 filter, and can still see some pinkish color on the right side of the image. Thank you.

— Reply to this email directly, view it on GitHub https://github.com/IntelRealSense/librealsense/issues/7155#issuecomment-1062173423, or unsubscribe https://github.com/notifications/unsubscribe-auth/AITQOVAB3XOKWYP7BHZWXBDU66ZERANCNFSM4QHYEMJQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you commented.Message ID: @.***>

[MartyG-RealSense]

@nievesbrunet If you are already using an ND filter and the camera is in a scene with bright sunlight then Intel provide additional advice about use in sunlight at the link below.

https://dev.intelrealsense.com/docs/tuning-depth-cameras-for-best-performance#section-use-sunlight-but-avoid-glare

[jcremona]

Hi @MartyG-RealSense @agrunnet Do you suggest applying the filter only in front of the color sensor or also on the IR cameras? Is the depth estimation affected?

[MartyG-RealSense]

Hi @jcremona For this particular RGB color hue issue, Intel advises that only the RGB sensor should have the filter attached to it.

It is fine to add ND filters over the IR sensors too though if you wish to do so, though ensure that both the left and right sensors have a filter fitted.

[jcremona]

After adding an ND8 filter, I still get a reddish colour on the left side (the filter helps but not enough). Also, the white lines have a strange effect, where they look like they are shifted or duplicated. Does anyone have a suggestion or possible explanation as to why this is happening?

[marianbrunet]

@jcremona I don't have an explanation, but I have seen a similar issue. Even after using the ND filter, the magenta shade persists. We had to switch to a different camera for our project.

[MartyG-RealSense]

In addition to adding an ND filter, Intel's official guidance about this D455 characteristic recommends that increasing the FPS speed may help to reduce the saturation effect further. So if 30 FPS is currently being used, please try 60 FPS at 848x480 RGB resolution to see whether it makes a positive difference.

[matheusSynkar]

Hello @MartyG-RealSense , thanks for the new information about the FPS. I tried and it works well but we checked the increasing of the frame lost in the Realsense Viewer. As we can solve the problem only increasing the FPS, it looks like that is a problem that can be solve by software. Is it Intel working in a firmware to help us to have the same result as the observed when increasing the fps but with less frame rate? Tks!

[MartyG-RealSense]

Hi @matheusSynkar As mentioned by other RealSense users above and in Intel's official guidance on page 8 (the final page) of the PDF document at the download link below, adding a Neutral Density (ND) filter over the camera lenses on the outside of the camera can help to reduce the impact of this issue.

https://www.intelrealsense.com/download/13629/?_ga=2.41245793.2144105902.1660733096-1041004708.1589641371

The phenomenon is not a bug but rather a characteristic of the D455 hardware and there are no plans to address it in firmware.

[matheusSynkar]

Hi @MartyG-RealSense , Thank you for the fast response. I understand your point, but in our tests with the d455 we were able to completely remove the problem by increasing it to 90fps without using an ND filter. In our point of view, it was solved by software and can be replicated in a firmware update. I'm attaching some images of tests done without ND filters.

[matheusSynkar]

Sorry for my persistence. I can't see why it wouldn't be possible to capture the same 90FPS and delivering 30FPS or 15FPS on the USB port. The camera system could discard the frames or for a more elegant solution, it would be possible to use the same shutter speed at 90FPS for lower FPS. The fastest shutter speed is the turning point. At a lower FPS we cannot achieve the same speed that 90FPS achieves.

[MartyG-RealSense]

I will raise your question with my Intel RealSense colleagues. Thanks very much for your patience!

[MartyG-RealSense]

Hi @matheusSynkar My colleagues suggest using manual exposure instead of auto-exposure and setting a shorter exposure time. An example of a shorter time for RGB exposure would be '78' instead of '156'.

In cases where the RGB FPS needs to be increased to deal with a shutter related problem, setting FPS to a minimum value of '6' (or 5 in the case of the D455 model) and a manual exposure of '78' can be an alternative to using a high FPS on auto-exposure.

[matheusSynkar]

Hello @MartyG-RealSense , Thank you for the helping to correct this issue. I tried the configuration as your colleague suggested. Please check the images below. As you can see, we have the same results in manual and automatic exposure. I think the minimum exposure time is not enough to correct the problem but with higher FPS the problem disappears.

15FPS 30FPS 60FPS 90FPS (It was necessary to reduce the image size due to the data transfer capability of the device)

[FPSychotic]

I think the best is return the camera as the first ones are faulty, the proof the revised ones have not such issue. Just return or complaint by a faulty product, there is enough info to proof that in this github

I returned 2 because that, intel should be recalled.

On Tue, 25 Oct 2022, 14:28 matheusSynkar, @.***> wrote:

Hello @MartyG-RealSense https://github.com/MartyG-RealSense , Thank you for the helping to correct this issue. I tried the configuration as your colleague suggested. Please check the images below. As you can see, we have the same results in manual and automatic exposure. I think the minimum exposure time is not enough to correct the problem but with higher FPS the problem disappears.

15FPS [image: 15fps] https://user-images.githubusercontent.com/107000971/197785704-d1cd6aa9-cb6c-4e08-9f5c-2237d1382183.png 30FPS [image: 30fps] https://user-images.githubusercontent.com/107000971/197785713-ab14fae5-9549-47a9-8710-f949b366a85a.png 60FPS [image: 60fps] https://user-images.githubusercontent.com/107000971/197785721-a854cbb3-1ddf-4b7d-bedc-63c9d26e34ac.png 90FPS (It was necessary to reduce the image due to the data transfer capability of the device) [image: 90fps] https://user-images.githubusercontent.com/107000971/197785735-585fee96-586b-4ade-8dfe-5d3da62f16b9.png

— Reply to this email directly, view it on GitHub https://github.com/IntelRealSense/librealsense/issues/7155#issuecomment-1290553584, or unsubscribe https://github.com/notifications/unsubscribe-auth/AITQOVGHKZPIOHHTXH5BJOLWE7OBVANCNFSM4QHYEMJQ . You are receiving this because you commented.Message ID: @.***>

[MartyG-RealSense]

Hi @matheusSynkar Please send me an email with your contact details included in the message.

[ArcadeHustle]

@MartyG-RealSense was there an over all solution to this "closed" case, that I missed?

[MartyG-RealSense]

Hi @ArcadeHustle This issue was not a bug but a characteristic of the camera hardware. https://github.com/IntelRealSense/librealsense/issues/7155#issuecomment-991084290 has a link to guidance about reducing the effect

[ArcadeHustle]

So a bug in the BOM none the less ;) that is unfortunate. Will there be a new revision of hardware with a different sensor? It is a bit unfortunate to learn about this post investment into several cameras in the model line. Can we expect a new model with a different sensor? Akin to what Luxonis is doing here by using IMX378? https://www.luxonis.com/blog/lens_color_filtering_enhancement

[MartyG-RealSense]

Whilst it is a hardware design characteristic and not a fault, D455 units manufactured after early 2022 have a design tweak that addresses the issue. So any D455 purchased now should have that revision.

[MAVProxyUser]

@MartyG-RealSense is there a way to query the hardware to check when it was manufactured?