D435i IMU calibration param

[w407022008]

Hello,

D435i, librealsense-2.50.0.

Is there a way to restore the imu calibration parameters to factory default settings only, without changing the calibration parameters of the camera set by the user?

BTW, even in the presence of severe vibrations, the mean value of the norm of the acceleration measurement vector should still be around 9.8 and not near 5 as I encountered, right? What I encountered was that when the drone was on the ground, the acceleration measurements were near 9.8, and after takeoff, the sliding window average value was near 5, even though I have added a foam damping pad.

Thx.

[MartyG-RealSense]

There is not an option to revert the IMU calibration to its original factory-new configuration.

The ideal Y-accel target value is 9.80, yes. Are you taking off / landing rapidly like a UAV in https://github.com/IntelRealSense/librealsense/issues/3991 that had problems with their RealSense IMU when doing so, please?

[w407022008]

well my issue is still a little different from this one. Usually when I hold UAV in hand, the slam system works robustly due to good imu data without serious vibration noise. However, once I installed it on the UAV, the vibration noise was very, very significant. This is understandable. So I added a foam pad to dampen the vibration. However, the problem was not solved, and then I recorded the imu data and found that the mean y-direction acceleration remained basically around 9.8 before takeoff and after landing, and around 5 after takeoff even when hovering in the air. Not long ago, the slam system was able to work properly even without adding foam pads. But at that time I did not pay attention to the imu measurements. Now it never works properly. So I'm not sure what could be causing the acceleration measurement error.

[w407022008]

Here is the log. The blue curve is the camera/imu and the red curve is the additional imu measurement. They all added a layer of foam pads to dampen the vibration. But camera/imu is not only noisier, but also seems to be shifted. But this does not happen when the uav lands.

[MartyG-RealSense]

If you are using a general foam pad not intended specifically for dampening then you may need a dedicated dampener material that is designed for vibration absorption, like the ones that can be placed on the feet of tripods. Searching a store such as Amazon for the term dampener pad will provide some useful results for example products.

The dampener pad can be placed at the point where the camera is mounted to the frame.

[w407022008]

So, do you think such a result is still caused by vibrations? Because I found that its center is shifted. And previously installed on the uav, slam was able to work properly. So I didn't exactly think it was due to vibration at first. But at the moment I don't have the imu data from the previous flight, so I can't be completely sure. I will try to use a new dampener. However, the point is that I am not entirely sure that this is due to vibration only.

[MartyG-RealSense]

Are you referring to the center of the depth image having shifted? Severe vibration can cause a mis-calibration of the non-IMU camera sensors such as depth. This can be corrected with a software re-calibration of the Depth and RGB sensors using the Dynamic Calibration software tool.

The tool can be downloaded for Windows at the link below.

https://www.intel.com/content/www/us/en/download/645988/intel-realsense-d400-series-dynamic-calibration-tool.html

Linux users can install the tool using instructions on page 14 onwards of the tool's PDF user guide.

https://www.intel.com/content/www/us/en/support/articles/000026723/emerging-technologies/intel-realsense-technology.html

[w407022008]

Oh no, I meant that the mean of the imu measurements on the y-axis is not around 9.8, but around 5. This is strange to me. Because normally the vibration should cause an approximate white noise. This is why I wonder if there is a problem with the acceleration measurement. The non-IMU sensors work always well, even though with severe vibration.
What do you think about such acceleration measurements, especially when the mean value is not around 9.8, like the red curve? I will also try to test it tomorrow with a special dampener pad.

[MartyG-RealSense]

Are you accessing the camera's IMU directly with your SLAM system and not using intermediate software such as the RealSense SDK or RealSense ROS wrapper to obtain the IMU values, please?

If you are accessing the IMU directly then the raw IMU data will be naturally noisy, as advised by a RealSense team member at https://github.com/IntelRealSense/librealsense/issues/4391#issuecomment-509921061

[w407022008]

No, actually i just used the ros topic. Well it's done, because exactly of the severe vibration. But it's also strange because it suddenly failed, whereas before it was feasible. Anyway it's done. In addition, If i accesse the imu directly, will I get the same frequency measurements as ros topic (65/250hz for accel, 200/400hz for gyro)?

[MartyG-RealSense]

You can access the IMU's raw data with the ROS package imu_filter_madgwick

http://wiki.ros.org/imu_filter_madgwick

Intel's D435i SLAM guide for ROS provides an example of using the IMU via this package.

https://github.com/IntelRealSense/realsense-ros/wiki/SLAM-with-D435i

I do not know the default frequencies that the raw data would use though if you were not publishing the IMU gyro and accel topics from the RealSense ROS wrapper.

[w407022008]

Hello @MartyG-RealSense , This package is noted. I will try it later. I have encountered a new problem. Whenever the drone lands, probably due to a non-severe shock, the camera communication goes down and the terminal displays the following.

01/06 05:40:46,647 ERROR [545997713776] (uvc-streamer.cpp:106) uvc streamer watchdog triggered on endpoint: 130
 01/06 05:40:46,651 ERROR [545971917168] (uvc-streamer.cpp:106) uvc streamer watchdog triggered on endpoint: 131
 01/06 05:40:46,687 WARNING [545997713776] (messenger-libusb.cpp:30) reset_endpoint returned error, index: 130, error: Device or resource busy, number: 16
 01/06 05:40:46,687 WARNING [545971917168] (messenger-libusb.cpp:30) reset_endpoint returned error, index: 131, error: Device or resource busy, number: 16
 01/06 05:40:46,687 WARNING [546593300848] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Device or resource busy, number: 16
 01/06 05:40:46,688 WARNING [546593300848] (messenger-libusb.cpp:92) usb_request_queue returned error, endpoint: 135 error: Device or resource busy, number: 16
 01/06 05:40:46,688 WARNING [545997713776] (messenger-libusb.cpp:92) usb_request_queue returned error, endpoint: 130 error: Device or resource busy, number: 16
 01/06 05:40:46,688 ERROR [545997713776] (uvc-streamer.cpp:138) failed to submit UVC request, error: -13
 01/06 05:40:46,688 WARNING [545997713776] (messenger-libusb.cpp:92) usb_request_queue returned error, endpoint: 130 error: Device or resource busy, number: 16
 01/06 05:40:46,688 ERROR [545997713776] (uvc-streamer.cpp:138) failed to submit UVC request, error: -13
 01/06 05:40:46,688 WARNING [545971917168] (messenger-libusb.cpp:92) usb_request_queue returned error, endpoint: 131 error: Device or resource busy, number: 16
 01/06 05:40:46,688 ERROR [545971917168] (uvc-streamer.cpp:138) failed to submit UVC request, error: -13
 01/06 05:40:46,688 WARNING [545971917168] (messenger-libusb.cpp:92) usb_request_queue returned error, endpoint: 131 error: Device or resource busy, number: 16
 01/06 05:40:46,688 ERROR [545971917168] (uvc-streamer.cpp:138) failed to submit UVC request, error: -13
 01/06 05:40:46,691 WARNING [546960081264] (messenger-libusb.cpp:92) usb_request_queue returned error, endpoint: 134 error: Device or resource busy, number: 16
 01/06 05:40:46,691 ERROR [546960081264] (hid-device.cpp:167) failed to submit UVC request

So what is usually the cause of this phenomenon? Thx.

[MartyG-RealSense]

Is it possible that the camera's USB cable is being shaken during the landing and causing a disconnection?

[w407022008]

I can not rule out this possibility, but usually plugging and unplugging usb cable is very hard to get, as opposed the fpc cable connecting the camera to the motherboard I think may be more likely to have poor contact, because I have removed the silver Aluminum case, which is too heavy. So usually the cause of this problem is poor contact with the data cable, or insufficient power supply, or both can cause such a phenomenon?

[MartyG-RealSense]

Are you using the FPC cable as the data transfer cable between camera and motherboard instead of using a USB cable?

You mentioned above that the non-IMU sensors were working normally. That suggests to me that the cable or the camera connection is not being disrupted / interfered.

Intel's D435i SLAM guide highlights that a sudden movement change can cause the IMU to lose tracking. I wonder whether a sudden stop when landing would count as such a change as velocity becomes zero. This section of the guide is quoted below.

---

The built-in IMU can only keep track for a very short time. Moving or turning too quickly will break the sequence of successful point cloud matches and will result in the system losing track. It could happen that the system will recover immediately if stopped moving but if not, the longer the time passed since the break, the farther away it will drift from the correct position. The odds for recovery get very slim, very quickly. The parameters set in the launch file are most likely not ideal but this is a good starting point for calibrating.

[w407022008]

Are you using the FPC cable as the data transfer cable between camera and motherboard instead of using a USB cable? Sorry I didn't express clearly. I used a usb3.0 cable connected with a D435i and an ARM based computer. The fpc cable is inside the D435i and connects the motherboard of the D435i with the native camera. Intel's D435i SLAM guide highlights that a sudden movement change can cause the IMU to lose tracking. It's not a slam problem, but the camera stream stopped and ros-wrapper stopped publishing to me camera messages and imu messages. I'm not sure exactly what caused such an interruption. Because it seems that it is not only during the landing phase that such problems occur, but sometimes in flight as well. However the arm computer that powers it is working fine, and the usb port of the arm computer can provide 5V 1.7A power, is that enough?

[MartyG-RealSense]

A RealSense team member once suggested budgeting for a maximum power draw of 2W per camera. This makes me think that your particular USB port's specification should be able to meet the power needs of the camera.

The flexible cable that connects the camera board to the board that the USB cable attaches to (the Vision Processor D4 Board) is called the interposer cable. At either end of the cable is a 50 pin connector. Errors can result if the connectors of the cable are not fully seated on the boards.

I note that you mentioned that you had removed the silver aluminum case, which would expose the interposer cable to the open air and any wind that was blowing at the time during the drone flight. I would speculate that the rapidly spinning propellors of a flight drone could also create turbulent air that could move against anything on the drone that was exposed to open air beneath the blades, perhaps making the interpower cable flap about. It would also be worth checking the operating temperature of the camera in the RealSense Viewer to see whether in-flight air movement was disrupting the airflow around the camera board and overheating it by making it more difficult for hot air to dissipate away naturally from the camera board.

A rigid circuit board version of the interposer can be purchased that would not be able to flex.

https://www.framos.com/en/products/d400-rigid-50-pin-interposer-20737

[w407022008]

Ok, let me have a test. It would also be worth checking the operating temperature of the camera in the RealSense Viewer to see whether in-flight air movement was disrupting the airflow around the camera board and overheating it by making it more difficult for hot air to dissipate away naturally from the camera board. So usually overheating could also be an important reason which cause this issue, right?

[MartyG-RealSense]

The official recommended maximum operating temperature is 35 degrees C, though in real-world conditions it can likely go up to 42 degrees C before problems begin manifesting above that point.

When the temperature reaches 60 degrees C or above and stays at that level for several seconds then a safety mechanism in the firmware driver will first halve the laser power value to try to bring the temperature down and then shut the laser off completely if the temperature continues to exceed 60 for several more seconds. If the laser is shut off then the camera can continue to operate, though the quality of the depth image may reduce.

Overheating could also be caused by a glitch in the USB port, a bad USB cable or overheating of a single-board computer that the camera is attached to that transfers the heat to the camera.

[w407022008]

OK, well.. I just tested it. There is no aluminum housing, the inner metal bracket is replaced with a 3d printed plastic bracket, the environment is almost windless, the viewer installed on windows shows an initial temperature of 30 degrees, in this case, the temperature of the asic chip quickly climbs to 50 degrees within 3 minutes and then slowly rises, about 10 minutes after starting the asic chip temperature exceeds 60 degrees. During the period, the temperature of the projector is slowly approaching 60 degrees. But it seems that even so, everything is still normal in the viewer. So, I think I should indeed add a heatsink to reduce the temperature of the asic chip. But does this seem to be the exact reason for that issue? I then used an electric fan to simulate the propeller wind, and both temperature values were basically maintained at about 45 degrees.

A rigid circuit board version of the interposer can be purchased that would not be able to flex. This version is batter than the fpc interposer? It just seems to me that the D4 board can only be side-by-side with the camera if we use such a rigid interposer. This seems to make its interface more prone to loosening due to vibration etc doesn't it? The interfaces on this rigid interposer are the same as the fpc interposer, right? The only difference is replacing the fpc cable with a pcb is it?

[MartyG-RealSense]

If the temperature is reaching 60 degrees or more then there is definitely a problem in your hardware setup that needs resolving. If there is not a very high natural environmental temperature in the location that the camera is being used in that could be heating up the camera then I would consider 45 degrees C or more a very problematic operating temperature.

A severe temperature event could also cause the camera's sensors to become mis-calibrated and need to be re-calibrated with software.

If the camera is becoming very hot within 3 minutes of starting from cold then this can be a classic symptom of there being an electrical problem in the USB port or the USB cable. For example, a USB cable could have become damaged internally from repeated bending.

The rigid interposer is as good as the flexible cable interposer and essentially the same as using a cable, though a rigid connector board is not suitable for every project, especially ones where the two boards need to be oriented in a particular way in relation to each other to make the best use of the available space.

[w407022008]

For example, a USB cable could have become damaged internally from repeated bending. Yes, I am using a flexible cable, similar to fpc. this possibility does exist. I just tested it and temporarily encountered no problems when using a standard usb3.0 cable, and encountered the issue mentioned before when using this flexible cable. So we can basically determine that such an issue is caused by internal damage of the usb cable, especially in the flight process severe vibration is very likely to cause contact failure. this can be a classic symptom of there being an electrical problem in the USB port or the USB cable However, it seems that the temperature of the asic chip rises quickly regardless of which cable is used. This then means that something has gone wrong with the usb port, right? I don't have an infrared thermal imager on hand, but when I touch it I find that the asic chip is still the highest temperature, while the nearby type c control chip, buck chip, etc. are warmer but lower than the asic chip. Is it necessary to replace the D4 board or just add a heatsink for the issue of overheating?

[MartyG-RealSense]

If the glitch that causes overheating is in the USB port then it would depend on which port has the glitch - the one on the computer or the one on the D4 board. You could test for whether it is the computer's port by plugging the camera into a different port if the computer has one or putting some kind of intermediary device such as a USB hub between the computer port and the camera, plugging the camera into the hub. I would think it more likely that it is the computer's port.

Another way to test the computer port would be to plug in a different USB device with a touchable casing and see whether that heats up rapidly too.

If you are using a single-board computer then they do sometimes overheat, so if the board has a cooling fan over its CPU then you could see whether the board supports increasing the fan speed. There is also the option of using a larger heatsink with the camera board, as you mentioned. There was also once a RealSense case where an apparently faulty power component of a single board computer was overheating and in turn heating up the camera.

[w407022008]

I tried different usb ports on several computers with a 1meter-long cable, including a separately powered usb hub. asic chip started to warm up still faster. However, since the ambient temperature has now dropped 6 degrees compared to before, the asic temperature is slow to warm up after reaching 40 degrees and for 10 minutes with a maximum temperature of no more than 60 degrees, while the maximum temperature of the projector is 42 degrees. So this looks like a problem with the usb port on the D4 board?

[MartyG-RealSense]

My understanding is that the laser safety mechanism of the firmware driver only takes the projector temperature into account and would not shut off the laser if the ASIC temperature exceeds 60 degrees.

Intel recommend a maximum casing temperature of 50 degrees C. If the silver outer casing is absent then the outer shell of the depth module represents the casing. There is a relationship between the casing temperature and the internal temperature. Intel lab tests found that the projector temperature should not exceed 60 degrees C if the casing temperature is below 44 degrees C. Therefore, external natural ambient environment temperature can have a bearing on internal projector temperature as the casing is heated by the environment.

If you have eliminated the computer and the USB cable as the cause then it does increase the likelihood that the problem lies with the D4 board's USB port.

Another factor that you could consider is your location's height above sea level. The higher you are, the thinner the air can be and the harder that cooling mechanisms have to work.

[w407022008]

I work in Paris, so I made sure there was enough air density to dissipate heat naturally. The temperature of the projector seems to be quite normal, as it climbs very slowly. Whereas the asic temperature does, no matter what the ambient temperature is,and at the beginning it will climb quickly to 50 degrees Celsius, and then slowly approach 60 degrees or even exceed. This seems to mean that the natural cooling power is exactly close to the heating power at 30 degrees difference from the ambient temperature. I will be purchasing a new usb cable to replace that flexible one and adding a heatsink to the asic chip. Anyway, thanks for your kind help and I will open it when I have the latest results, and if there is still same issue. Thanks again.

[MartyG-RealSense]

Thanks very much for the update. I look forward to your next test results. Good luck!

[w407022008]

Hello @MartyG-RealSense

The heat sink is effective especially in flight.

The main problem of data loss is indeed caused by the flexible usb cable. Perhaps because the internal failure of the cable is reflected in the flight due to violent vibrations. I have replaced one and it is working fine.

I still want to replace the D4 board. Is this the D4 board used in the d435i? Its CODE is 82635DSASICBRDI, and 999AFP.

[MartyG-RealSense]

It's excellent to hear that you achieved significant improvement after component changes.

Yes, the V2 version of the Vision Processor D4 Board with part number 999AFP that you linked to is used with D435i. V3 is for D450 / D455 and V4 is for D401 / D405. V1 was for D415 and D435.

https://www.framos.com/en/products/vision-processor-d4-v2-imu-version-24794