D435i - IMU - linear acceleration in z without any movement

[BarzelS]

Hi, I'm getting linear acceleration in z axis even without any movement of the camera. Is it normal? I think thats the reason my complementary filter drift when calculating the euler angles(rpy).

The values of z are constantly about 0.24

(The camera is steady on a table without any movement)

rostopic echo /camera/imu :

header: 
  seq: 6324
  stamp: 
    secs: 1594727089
    nsecs: 860252142
  frame_id: "camera_imu_optical_frame"
orientation: 
  x: 0.0
  y: 0.0
  z: 0.0
  w: 0.0
orientation_covariance: [-1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
angular_velocity: 
  x: -0.0
  y: -0.00349065847695
  z: 0.00174532923847
angular_velocity_covariance: [0.01, 0.0, 0.0, 0.0, 0.01, 0.0, 0.0, 0.0, 0.01]
linear_acceleration: 
  x: -0.0882598459721
  y: -9.40457725525
  z: 0.245166242123

rqt plot:

[MartyG-RealSense]

Hi @shirbarzel I see from your recent case about the IMU that you have a D435i. Have you performed a calibration on the IMU please?

The D435i's IMU hardware component does not have internal calibration. So unless it is calibrated with an IMU calibrator tool (separate from tools used to calibrate other sensors such as depth and RGB), vertical acceleration may be unable to settle at zero even if the camera is not moving.

The calibrator tool can be found here:

https://github.com/IntelRealSense/librealsense/tree/development/tools/rs-imu-calibration#rs-imu-calibration-tool

Intel's guide to getting data from the IMU provides the following information on the subject:

https://www.intelrealsense.com/how-to-getting-imu-data-from-d435i-and-t265/

---

The IMU sensor does not include internal calibration, which may manifest itself with non-zero angular velocities produced at idle mode and the gravity (accel) force measured being not equal to 9.80665 m/s2.

For D435i the IMU sensor is not calibrated, hence a free calibration tool is available as as part of the SDK. Running the calibration routine will calculate IMU intrinsic covariance and zero offsets, and then store them on device’s NVRAM for later use in SDK.

The depth<->IMU sensor extrinsic (rigid body transformation) is precalculated based on mechanical drawings and cannot be modified. When initialized, the SDK will query the device for the existence of the IMU calibration data, and if present – apply it to the raw IMU samples produced by the device.

[BarzelS]

I already perform the calibration using the rs-imu-calibrtation-tool a month ago. But I can try it again. Just to understand, does values like this:

linear_acceleration: 
  x: -0.0882598459721
  y: -9.40457725525
  z: 0.245166242123

While no movement on the table are normal? or do i expect to get 0.0 in the z acceleration after the calibration?

Thanks @MartyG-RealSense!

[MartyG-RealSense]

Ideally, you want Y to be as close to 9.8 as possible (the value of the Earth's gravity). I recall though that some people have had values of around 9.4 even after IMU calibration. RealSense programs that can make use of motion correction can adjust the raw IMU value though so that it is closer to 9.8.

Please see the linked-to comment below and the comment below it.

https://github.com/IntelRealSense/librealsense/issues/4751#issuecomment-526344423

[BarzelS]

Ideally, you want Y to be as close to 9.8 as possible (the value of the Earth's gravity). I recall though that some people have had values of around 9.4 even after IMU calibration. RealSense programs that can make use of motion correction can adjust the raw IMU value though so that it is closer to 9.8.

Please see the linked-to comment below and the comment below it.

IntelRealSense/librealsense#4751 (comment)

1. I was trying to calibrate using the rs-calibration-imu.py and got this message:

Writing calibration to device.

Error: failed to set power state
Segmentation fault (core dumped)

Any idea why?

2. You answered me regarding the acceleration in Y axis but what about Z axis? what to expect?

[MartyG-RealSense]

1. I have only seen one case where the failed to set power state error appeared as a result of trying to write the IMU calibration result to eeprom, and an investigation didn't find a solution for it, unfortunately.

https://github.com/IntelRealSense/librealsense/issues/6370

Do you have another computer that you can try the calibration on, please? As the result is stored inside the camera, a successfully completed calibration should be carried back to any other computer that you plug the camera into.

2. For comparison I laid my uncalibrated-IMU D435i on a flat surface and tested the accel values to give you some values to compare to your own.

On average I got 0.2 to 0.4 for Z accel when placing the D435i on a range of flat surfaces, but never absolute zero.

[BarzelS]

1. I have only seen one case where the failed to set power state error appeared as a result of trying to write the IMU calibration result to eeprom, and an investigation didn't find a solution for it, unfortunately.

IntelRealSense/librealsense#6370

Do you have another computer that you can try the calibration on, please? As the result is stored inside the camera, a successfully completed calibration should be carried back to any other computer that you plug the camera into.

1. For comparison I laid my uncalibrated-IMU D435i on a flat surface and tested the accel values to give you some values to compare to your own.

On average I got 0.2 to 0.4 for Z accel when placing the D435i on a range of flat surfaces, but never absolute zero.

1. I hope I managed to fix it with a different computer, at the end it wrote:

   
   Writing calibration to device.
   Device PID:  0B3A
   Device name:  Intel RealSense D435I
   Serial number:  845112070914
   Firmware version:  05.12.06.00
   SUCCESS: saved calibration to camera.
   Done.
   Segmentation fault (core dumped)

So I hope it actually success even tough there was a segmentation fault

2. Now I'm still getting this values:

angular_velocity: x: 2.81880511466e-05 y: 7.24156052456e-05 z: 1.14160184239e-06 angular_velocity_covariance: [0.01, 0.0, 0.0, 0.0, 0.01, 0.0, 0.0, 0.0, 0.01] linear_acceleration: x: 0.345232009888 y: -9.79773902893 z: 0.230222553015

You said your values were with an uncalibrated-IMU D435i  what are the values for a calibrated IMU?

3. BTW, I have just checked the rs-motion and I'm not getting any yaw while doing yaw rotation to my camera, is it related?

[MartyG-RealSense]

I'm not able to calibrate my own D435i IMU right now and I couldn't find a good example of a table of values from a calibrated IMU that includes x and z. My research strongly indicates though that the values of x and z at idle are less important than y being as near to 9.8 as you can achieve.

[BarzelS]

I'm not able to calibrate my own D435i IMU right now and I couldn't find a good example of a table of values from a calibrated IMU that includes x and z. My research strongly indicates though that the values of x and z at idle are less important than y being as near to 9.8 as you can achieve.

1. Thanks, please let me when you will have this kind of table, I need to get some feedback regarding if my calibration was correct.
2. What about 3. in my comment above?
3. How often should I perform calibration to my camera?

Thanks

[MartyG-RealSense]

1. I did further extensive research for a table of post-calibration accel values but was again unable to find any references. x and z does not seem to be something that can be easy to measure compared to the y value. For example, the values may be affected during calibration depending on the orientation of the camera or how flat and level the surface that the camera is resting on is.

2. A couple of other RealSense user have reported little or no yaw in rs-motion in past cases. A RealSense team member thought that there may be something wrong with the gyro and suggested testing the IMU in the RealSense Viewer's motion module.

https://github.com/IntelRealSense/librealsense/issues/4391#issuecomment-526034550 https://github.com/IntelRealSense/librealsense/issues/4915

3. I don't think that there is an official recommendation for how often to recalibrate. If the camera receives a hard knock or a drop on the floor then it may be worth doing a total recalibration (IMU and non-IMU sensors) if it looks as though the output image is broken / noisy or less accurate than it was previously.

If the camera is being used in an application where accuracy is critically important, such as in an industrial or commercial setting, then recalibrating all sensors once a week as part of a regular maintenance cycle would help to keep the camera's accuracy optimised.

[BarzelS]

1. I did further extensive research for a table of post-calibration accel values but was again unable to find any references. x and z does not seem to be something that can be easy to measure compared to the y value. For example, the values may be affected during calibration depending on the orientation of the camera or how flat the surface that the camera is resting on is.
2. A couple of other RealSense user have reported little or no yaw in rs-motion in past cases. A RealSense team member thought that there may be something wrong with the gyro and suggested testing the IMU in the RealSense Viewer's motion module.

IntelRealSense/librealsense#4391 (comment) IntelRealSense/librealsense#4915

1. I don't think that there is an official recommendation for how often to recalibrate. If the camera receives a hard knock or a drop on the floor then it may be worth doing a total recalibration (IMU and non-IMU sensors) if it looks as though the output image is broken / noisy or less accurate than it was previously.

If the camera is being used in an application where accuracy is critically important, such as in an industrial or commercial setting, then recalibrating all sensors once a week as part of a regular maintenance cycle would help to keep the camera's accuracy optimised.

1. Thanks.
2. So following question 1. the only way to check it is with the rs-motion, I have seen already this posts that you attached but I haven't found any answers to my problem, I'm having this problem in both my PC and my Jetson TX2 so I don't think it's related, any other suggestions how to get the rs-motion works correctly?
3. Thanks

[MartyG-RealSense]

Aside from using the Motion Module of the RealSense Viewer, you can also use the rs-multicam example to check the D435i's IMU streams. It creates a quantity of stream panels relating to the number of cameras attached. So despite the program's name, it still works with a single camera.

[BarzelS]

Aside from using the Motion Module of the RealSense Viewer, you can also use the rs-multicam example to check the D435i's IMU streams. It creates a quantity of stream panels relating to the number of cameras attached. So despite the program's name, it still works with a single camera.

1. Thanks @MartyG-RealSense, I tried it but I don't understand how it helps me validate that my IMU streams are correct:

2. BTW, regardless the IMU calibration are there any other calibration needed for the cameras also? how often should I do them and how?

[MartyG-RealSense]

1. Every reference source that I know of, including the official calibration instructions in the documentation and the program itself, places the primary importance on getting a y accel value as close to 9.8 as possible and barely mentions the x and z values. Unless you are getting obviously erratic measurements from your IMU then I would not be too concerned.

2. Aside from the IMU calibration, the depth and RGB can also be calibrated. You have two options here:

• The Dynamic Calibrator software. Windows users can download the software here:

https://downloadcenter.intel.com/download/29618/Intel-RealSense-D400-Series-Dynamic-Calibration-Tool

For Linux users, installation instructions are on page 14 onward of the software's user guide:

https://www.intel.com/content/www/us/en/support/articles/000026723/emerging-technologies/intel-realsense-technology.html

• The On-Chip Self Calibration built into the RealSense Viewer software. It can be accessed by going to the More option at the top of the Viewer's options side-panel and selecting On-Chip Calibration from its menu.

A guide to using the On-Chip Calibration is here:

https://dev.intelrealsense.com/docs/self-calibration-for-depth-cameras

[BarzelS]

1. Every reference source that I know of, including the official calibration instructions in the documentation and the program itself, places the primary importance on getting a y accel value as close to 9.8 as possible and barely mentions the x and z values. Unless you are getting obviously erratic measurements from your IMU then I would not be too concerned.
2. Aside from the IMU calibration, the depth the RGB can also be calibrated. You have two options here:

• The Dynamic Calibrator software. Windows users can download the software here:

https://downloadcenter.intel.com/download/29618/Intel-RealSense-D400-Series-Dynamic-Calibration-Tool

For Linux users, installation instructions are on page 14 onward of the software's user guide:

https://www.intel.com/content/www/us/en/support/articles/000026723/emerging-technologies/intel-realsense-technology.html

• The On-Chip Self Calibration built into the RealSense Viewer software. It can be accessed by going to the More option at the top of the Viewer's options side-panel and selecting On-Chip Calibration from its menu.

A guide to using the On-Chip Calibration is here:

https://dev.intelrealsense.com/docs/self-calibration-for-depth-cameras

1. Ok, thanks.

2. I'm trying to use the On-Chip Calibration but it's a bit confusing from the documentation what suppose to be the observed scene, it is OK to put the camera in about 0.8m away from a blank wall? (both regarding the intrinsic and the extrinsic calibration).

[MartyG-RealSense]

The Chief Technical Officer of the RealSense Group at Intel (agrunnet) offers some instructions for doing a self calibration here:

https://github.com/IntelRealSense/librealsense/issues/5838#issuecomment-598310831

[BarzelS]

1. I did further extensive research for a table of post-calibration accel values but was again unable to find any references. x and z does not seem to be something that can be easy to measure compared to the y value. For example, the values may be affected during calibration depending on the orientation of the camera or how flat the surface that the camera is resting on is.
2. A couple of other RealSense user have reported little or no yaw in rs-motion in past cases. A RealSense team member thought that there may be something wrong with the gyro and suggested testing the IMU in the RealSense Viewer's motion module.

IntelRealSense/librealsense#4391 (comment) IntelRealSense/librealsense#4915

1. I don't think that there is an official recommendation for how often to recalibrate. If the camera receives a hard knock or a drop on the floor then it may be worth doing a total recalibration (IMU and non-IMU sensors) if it looks as though the output image is broken / noisy or less accurate than it was previously.

If the camera is being used in an application where accuracy is critically important, such as in an industrial or commercial setting, then recalibrating all sensors once a week as part of a regular maintenance cycle would help to keep the camera's accuracy optimised.

1. Thanks.
2. So following question 1. the only way to check it is with the rs-motion, I have seen already this posts that you attached but I haven't found any answers to my problem, I'm having this problem in both my PC and my Jetson TX2 so I don't think it's related, any other suggestions how to get the rs-motion works correctly?
3. Thanks

Thanks @MartyG-RealSense But still the fact that the rs-motion doesn't recognize any yaw motion wouldn't hint that there is a problem with the IMU?

[MartyG-RealSense]

If you enable the Motion Module in the RealSense Viewer and move your mouse cursor over the Gyro panel then it should display changing text values for XYZ.

[MartyG-RealSense]

Hi @shirbarzel Do you require further assistance with this case please, or can it be closed now? Thanks!

[BarzelS]

Hi @shirbarzel Do you require further assistance with this case please, or can it be closed now? Thanks!

Yes, still my IMU is very noisy, as you can see here: https://github.com/HKUST-Aerial-Robotics/VINS-Fusion/issues/36#issuecomment-663484868

The user reported that he got (0,0,9.8) which is something I'm not getting doesn't matter how many calibrations I will do.

[MartyG-RealSense]

The documentation for the rs-motion program has some interesting commentary in its code. It says that increasing the alpha value can give more weight to the gyro when calculating theta (the angle of camera rotation in x, y and z components) but can cause drift if set too high. Reducing the alpha value meanwhile gives more weight in the theta calculation to the accelerometer, which is more sensitive to disturbances.

https://dev.intelrealsense.com/docs/rs-motion#section-code-overview

---

// theta is the angle of camera rotation in x, y and z components float3 theta; std::mutex theta_mtx; / alpha indicates the part that gyro and accelerometer take in computation of theta; higher alpha gives more weight to gyro, but too high values cause drift; lower alpha gives more weight to accelerometer, which is more sensitive to disturbances / float alpha = 0.98; bool first = true; // Keeps the arrival time of previous gyro frame double last_ts_gyro = 0;

---

The rs-motion program also applies a Complementary Filter to balance gyro and accelerometer.

[MartyG-RealSense]

Closed due to no further comments received.

[lishanggui]

@DronistB Have you solved the problem? I have a problem similar to yours. How did you solve it?

[BarzelS]

@DronistB Have you solved the problem? I have a problem similar to yours. How did you solve it?

Not really @lishanggui, what's your email? lets discuss there

[lishanggui]

OK,thank you ! my email is lishanggui33@126.com!

[MonchoG]

@SBarzz I am facing similar issue. Did you manage to solve your problem ?

[hzlijianjun]

I incurred same issue, but I saved calibration to camera successfully by luck. After six positions calibration procedures, I saved the calibration data to local file successfully, but fail to write to D435i device. Desperatly, I try writing the calibration data to device directly via : python rs-imu-calibration.py -i accel_<your footer>.txt gyro_<footer>.txt and It output:

...
SUCCESS: saved calibration to camera.
Done.

[MartyG-RealSense]

Thanks very much @hzlijianjun for sharing the method that worked for you!