D455 Linux depth variation over time

[rghl3]

---

Required Info
Camera Model	D455
Firmware Version	()
Operating System & Version	Ubuntu 18.04
Kernel Version (Linux Only)
Platform	PC
SDK Version
Language	C++
Segment

Issue Description

Hi, I have a d455 camera that's fit to look down the floor. I ran it for 12 hours and calculated the floor depth 5 minutes once for same region of interest with no change in external lighting conditions. Starting depth value: 1700mm Depth value after 12 hours: 1715mm The values kept on increasing from 1700 to 1715mm for 12 hours. Is there any possibility to avoid such behaviour?

Thanks.

[MartyG-RealSense]

Hi @rghl3 If the camera position and the lighting conditions remain constant, are there changes in the temperature during that 12 hour period (the air temperature of the environment that the camera is in or the internal ASIC and Projector temperatures of the camera).

[rghl3]

Hi @MartyG-RealSense, The projector temperature has varied from 35 to 41 degrees and asic temperature from 42 to 45 degrees.

[MartyG-RealSense]

Whilst your ASIC and Projector temperatures are very high, they are probably not high enough to introduce a hardware malfunction that could account for the measurement increases over time.

Your description suggests that it is a linear, predictable increase in distance error over time rather than random fluctuations.

If you have consistent lighting conditions 24 hours a day in the location that the camera is used in then you could try disabling auto-exposure and use a fixed manual exposure value (such as the camera's default depth exposure of '8500') to see whether it provides more stable measurement over long-term periods.

Another option would be to perform a periodic automated reset of the camera with the hardware_reset() instruction, either before each 5-minute capture or a less frequent period such as hourly.

[rghl3]

Hi @MartyG-RealSense ,

use a fixed manual exposure value (such as the camera's default depth exposure of '8500') to see whether it provides more stable measurement over long-term periods.

My use case requires the camera to be placed at any location over a longer period of time. So I am not sure of how much the manual exposure would affect the depth output. Nevertheless, I will try it and confirm if it provides stable measurement.

Your description suggests that it is a linear, predictable increase in distance error over time rather than random fluctuations.

I could observe a behavior where the depth value decreases when the temperature increases and vice versa.

Projector temperature( in degree celsius)	Depth value(in mm)
35	1688
28	1693

Is there any way to avoid this behavior or is this expected?

[MartyG-RealSense]

If your use case requires the camera to be placed in any location and operate over a long-term period then it would be best to have auto-exposure enabled in order to automatically adjust to the current environmental conditons.

I have not seen a case of a distance measurement rise and fall that corresponds predictably with a change in temperature.

The mounting position that you describe (looking down to the floor) suggests that the camera is used in an indoor environment and so high environmental temperature (air temperature) is not present in the locations where the camera will be installed. Can you confirm this, please?

[rghl3]

The mounting position that you describe (looking down to the floor) suggests that the camera is used in an indoor environment and so high environmental temperature (air temperature) is not present in the locations where the camera will be installed. Can you confirm this, please?

Yes, the camera is used in an indoor environment usually with room temperature of 25 to 35 degree celsius.

[MartyG-RealSense]

Thank you very much. The indoor room temperature could be classified as ranging from hot to very hot then.

The data sheet document for the 400 Series cameras states about exceeding 35 degrees C: "The Intel RealSense Depth Camera D400 Series camera can function above the 35 degrees C max ambient temperature when powered with degradation in performance and no guarantee of the camera meeting the depth quality metric".

This seems to be a situation where an active cooling solution external to the camera would work best. The data sheet document has the following commentary about this subject:

"While heat pipes and liquid cooling solutions offer great advantages, they also increase the complexity of system integration substantially. The use of a fan with direct air flow to the depth camera (back mounting surface 2xM3 screws) is preferable".

It should be noted that the back mounting holes on the D455 are a larger M4 size.

[MartyG-RealSense]

Hi @rghl3 Do you require further assistance with this case, please? Thanks!

[rghl3]

Hi @MartyG-RealSense , I got held up in a different task.Hence, the delay in response.

The Intel RealSense Depth Camera D400 Series camera can function above the 35 degrees C max ambient temperature when powered with degradation in performance and no guarantee of the camera meeting the depth quality metric.

I tested d415 camera and observed the same behaviour of increasing depth values with decreasing temperatures. I started at 30 degree celsius and cooled the camera down till 10 degree celsius. I could observe the depth values increasing as temperature reduced. I would like to know if there is a possibility for me to maintain the same depth value within the operating temperature range.

[MartyG-RealSense]

If your target depth value is 1700 mm, and during your earlier test it became 1688 mm when temperature increased to 35 degrees C and 1693 when temperature decreased to 28 degrees C, this suggests that depth error increases as temperature increases. Can you confirm whether this is correct, please?

It is worth noting that there will be a certain margin of error in the depth measurements, as error increases linearly over distance (a principle called RMS error). Information about RMS error can be found in the section of Intel's camera tuning guide linked to below.

https://dev.intelrealsense.com/docs/tuning-depth-cameras-for-best-performance#section-verify-performance-regularly-on-a-flat-wall-or-target

However, ideally you do want to minimize the possibility of the margin of error in the measurement increasing further due to temperature rise. As you are using the camera in an indoor environment with varying room temperatures of 25 to 35 degrees C (ranging from hot to very hot), this suggests that the cooling solution should either be one that is constantly powerful or increases its cooling efforts as temperature increases (like a computer cooling fan that spins harder as operating temperature rises).

What method did you use to cool the camera to 10 degrees please?

[rghl3]

If your target depth value is 1700 mm, and during your earlier test it became 1688 mm when temperature increased to 35 degrees C and 1693 when temperature decreased to 28 degrees C, this suggests that depth error increases as temperature increases. Can you confirm whether this is correct, please?

Yes.I could observe the above behavior.

What method did you use to cool the camera to 10 degrees please?

I mistakenly mentioned as cooling the camera. I cooled down the room temperature to 10degrees and not the camera. The camera projector temperature is almost 10 degree Celsius hotter than the room temperature. So I assume the camera projector temperature to be 20 degrees, during the above test.

It is worth noting that there will be a certain margin of error in the depth measurements, as error increases linearly over distance (a principle called RMS error)

The above behaviour was observed even for a height of 1000mm.

[MartyG-RealSense]

If the measured depth error at a certain temperature range is always consistent then you could read the camera's temperature with program scripting and apply a correction. For example, if the measurement at 35 degrees is 1715 mm instead of the correct 1700 mm then the rough logic could be:

if projector temperature > 30 then depth_mm_corrected = depth_mm - 15

[rghl3]

Since multiple depth levels are involved, we don't have sufficient data for how much depth varies at each level. So 1715 instead of 1700 may be 1305 at 1300mm. Is there a specific projector temperature range within which the camera performance doesn't degrade if temperature is varied within that range?

[MartyG-RealSense]

Essentially this is an environmental problem that is external to the camera hardware. As discussed earlier, it therefore could benefit from some kind of custom cooling / refrigeration solution that surrounds the camera or is mounted adjacent to it.

Intel have validated the camera for projector temperatures up to 35 degrees C. Beyond that temperatures, no guarantees about performance and depth quality can be made. So if predictable performance is vital then assuming 35 degrees to be the ideal maximum projector temperature to avoid exceeding if possible is recommendable.

How is the camera mounted in the face-down position? Is it on the end of a robot arm?

[rghl3]

We have mounted the camera on a vertical pole as of now and fit the camera facing down.

[MartyG-RealSense]

Is the pole metal? If it is, that could be conducting heat from the room and passing it to the camera through the mounting point in addition to the heat that the camera casing absorbs. Is the bottom of the pole low enough to touch with the hand and feel how hot it is?

[MartyG-RealSense]

SDK version 2.43.0 has just been released. For the D455 it adds "a compensation mechanism intended to mitigate the effect of thermal propagation in optics. When active (default = On) it will track and adjust Depth and RGB calibration parameters automatically".

https://github.com/IntelRealSense/librealsense/releases/tag/v2.43.0

Edit: 2.43.0 adds a RS2_EXTENSION_CALIBRATION_CHANGE_DEVICE instruction. "Extension API allowing to register user-defined callbacks for notifications of Thermal adjustments invocation. These callbacks are informative in their nature and can be used for environmental monitoring".

https://github.com/IntelRealSense/librealsense/wiki/API-Changes#version-2430

[rghl3]

Is the bottom of the pole low enough to touch with the hand and feel how hot it is?

The pole is made of a metal.The pole doesn't seem to be hot enough to feel.

SDK version 2.43.0 has just been released

I will try this version and update you if this solves.

[MartyG-RealSense]

Hi @rghl3 Do you have an update that you can provide about this case, please? Thanks!

[rghl3]

I am yet to test the above. I will close and re-open once I go back to that.

[MartyG-RealSense]

Okay @rghl3 that is no problem to re-open later if you need to. Thanks for the update!

[rghl3]

Hi @MartyG-RealSense,

SDK version 2.43.0 has just been released

The above sdk version seems to solve the varying depth values at different temperatures to a considerable extent.

The same behavior occurs for d415 camera also(increasing depth with decreasing temperature and vice versa). The release note mentions thermal optimization works only for d455.

Is there any possibility of thermal optimization, working for d415 camera also?

[MartyG-RealSense]

I am not familiar with the mechanics of the Thermal Compensation feature. In one of the links associated with the feature in the release notes though, mention is made of only the D455 having a thermal monitor.

[rghl3]

Is there a possibility in future for Thermal compensation feature to be included in d415 also?

[MartyG-RealSense]

I will seek advice from my Intel RealSense colleagues about your question @rghl3

[MartyG-RealSense]

Hi @rghl3 Bearing in mind the above comment that there is no further information that can be provided regarding your question, do you require further assistance with this case, please? Thanks!

[MartyG-RealSense]

Case closed due to no further comments received.