Problem with D435i imu calibration

lishanggui commented 4 years ago

Required Info
Camera Model	D435i
Firmware Version	05.12.01.00
Operating System & Version	Ubuntu 16.04
Kernel Version (Linux Only)	(e.g. 4.15.0)
Platform	PC
SDK Version	2.30.0
Language	C++

I used D435i sensor box as recommended in "" librealsense / tools / rs-imu-calibration / images / "to ensure that the posture's horizontal / vertical alignment is tested. The D435i was calibrated multiple times with the librealsense tool rs_imu_calibration.py. (I repeated all experiments in # 5177, and it has the same problem)

One calibration result is as follows:

Direction data collected.
Would you like to save the raw data? Enter footer for saving files (accel_<footer>.txt and gyro_<footer>.txt)
Enter nothing to not save raw data to disk. >

Not writing to files.
[-0.0006397  -0.00545291 -0.00237064]
[1000 1000 1000 1000 1000 1000]
using 6000 measurements.
[[  1.01317040e+00   3.22186653e-03  -3.26252919e-02]
 [  9.54452414e-04   1.02251739e+00  -1.98976802e-03]
 [ -6.86104605e-04  -7.11080569e-03   1.00041354e+00]
 [  1.25330453e-01  -2.07885656e-02   3.93350259e-01]]
residuals: [ 47.96966542   9.04185781  19.83116716]
rank: 4
singular: [ 443.37554228  429.08813651  425.47837306   77.24639572]
norm (raw data  ): 9.671779
norm (fixed data): 9.779570 A good calibration will be near 9.780300
Would you like to write the results to the camera's eeprom? (Y/N)y
Writing calibration to device.
  Device PID:  0B3A
  Device name:  Intel RealSense D435I
  Serial number:  912112071569
  Firmware version:  05.12.01.00
SUCCESS: saved calibration to camera.
Done.

However, right after this when I check the acc. Output in realsense-viewer, it is not close to 9.7803 (the local acceleration of gravity is 9.7803).

I tested with realsense-viewer, The orientation of the device is set to 6 different cases according to the calibration white paper. (Section 4.3.2)

Position 1:   9.759(motion correction on)    9.535(motion correction off)
Position 2:  10.009(motion correction on)    9.790(motion correction off)
Position 3:   9.792(motion correction on)    9.560(motion correction off)
Position 4:   9.542(motion correction on)    9.535(motion correction off)
Position 5:  8.964(motion correction on)    9.359(motion correction off)
Position 6:   10.568(motion correction on)   10.170(motion correction off)

In rs-motion.cpp main function, I added a couple of lines to check the value in the terminal as below.

// Get accelerometer measures
rs2_vector accel_data = motion.get_motion_data ();

the result:

Position 1: 9.75401
Position 2: 10.0678
Position 3: 9.81807
Position 4: 9.53813
Position 5: 8.98368
Position 6: 10.5537

Run 'rs-enumerate-devices -c' and the result is as follows

Motion Intrinsic Parameters:

Motion Intrinsic of "Gyro"    MOTION_XYZ32F
Bias Variances:     0.000000000000000  0.000000000000000  0.000000000000000  
Noise Variances:    0.000000000000000  0.000000000000000  0.000000000000000  
Sensitivity : 
     1.000000       0.000000       0.000000      -0.000011  
     0.000000       1.000000       0.000000      -0.000095  
     0.000000       0.000000       1.000000      -0.000041  

Motion Intrinsic of "Accel"   MOTION_XYZ32F
Bias Variances:     0.000000000000000  0.000000000000000  0.000000000000000  
Noise Variances:    0.000000000000000  0.000000000000000  0.000000000000000  
Sensitivity : 
     1.013170       0.000954      -0.000686       0.125330  
     0.003222       1.022517      -0.007111      -0.020789  
    -0.032625      -0.001990       1.000414       0.393350

The accelerometer's sensitivity matrix shows a notable bias for the third axis compared to the first two.

How can the problem be solved? Did the engineer reproduce the problem and get it resolved? I am currently using realsense D435i to do slam experiments, and I am very anxious to solve this problem.

patbenatar commented 4 years ago

I'm experiencing what sounds like the same issue.

After multiple attempts at python rs-imu-calibration.py using a rigid frame built and verified with a level, I'm getting accelerations notably deviating from gravity in certain orientations as well as significant values on the axes that I would expect to be near-zero.

Calibration

Motion Intrinsic Parameters:

Motion Intrinsic of "Gyro"    MOTION_XYZ32F
Bias Variances:     0.000000000000000  0.000000000000000  0.000000000000000
Noise Variances:    0.000000000000000  0.000000000000000  0.000000000000000
Sensitivity :
     1.000000       0.000000       0.000000       0.000001
     0.000000       1.000000       0.000000       0.000008
     0.000000       0.000000       1.000000      -0.000041

Motion Intrinsic of "Accel"   MOTION_XYZ32F
Bias Variances:     0.000000000000000  0.000000000000000  0.000000000000000
Noise Variances:    0.000000000000000  0.000000000000000  0.000000000000000
Sensitivity :
     1.018967      -0.004261       0.000489      -0.012745
     0.034915       1.027532      -0.005286       0.206483
    -0.034412      -0.004820       1.010221       0.472612

Observed accelerations

Position 1

x: -0.0824612751603
y: -9.84611606598
z: 0.943550467491

Position 2

x: 9.77985572815
y: -0.669295370579
z: 0.854578495026

Position 3

x: 0.0589955076575
y: 9.81116771698
z: 0.994111776352

Position 4

x: -9.82306098938
y: 0.695441424847
z: 1.07413542271

Position 5

x: -0.00596596579999
y: 0.0567697882652
z: -8.84863853455

Position 6

x: -0.0373049341142
y: -0.0409596562386
z: 10.7277708054

Any help for those of us struggling with D435i calibration would be much appreciated!

lishanggui commented 4 years ago

The rs_imu_calibration.py provided by Intel realsense requires that each position (write paper) needs to be completely horizontal or vertical during the calibration process. It is difficult to achieve absolute horizontal or vertical experimental conditions, and the accuracy of the calibration will be affected. Recently, I used other methods for calibration, and wrote the calibration results to the device through the rs_imu_calibration.py writer. I found that the results output using realsense-viewer and rs-motion are the same as above, and I suspected that it was an input device There was a problem with the post-correction imu data conversion process. Then I did this kind of experiment, let the equipment output the imu raw data, and use the calibration data to transform it by a program written by myself. As a result, the equipment was placed in a different way and the gravity acceleration was 9.8. Therefore, it is concluded from these experiments that the main reason for the inaccurate results after imu calibration is the problem of the underlying conversion. At the same time, I think that rs_imu_calibration.py is not a particularly good method, because reducing the accuracy requires more experimental equipment Horizontal and vertical.

ev-mp commented 4 years ago

@lishanggui hello, Thank you for the insights. Obviously the calibration precision is limited as it lacks ground truth and shifts this responsibility to the user. Thus the main purpose of rs_imu_calibration.py is to minimize the errors by adjusting both for the intrinsic and extrinsic sensor matrix.

And even in case of external equipment used to verify horizontal/vertical alignment there would still remain fundamental questions unresolved, such as to what degree the acceleration vector, lets say, is skewed from the plumb line due to its mechanical posture (mechanical) rather than the sensor's analog/digital bias. (The equipment can physically measure the orientation of the camera's case but not the IMU sensor)

Additional comments:

I found that the results output using realsense-viewer and rs-motion are the same as above, and I suspected that it was an input device There was a problem with the post-correction imu data conversion process

Note that writing the IMU calibration to the flash is only the first step. and to use the motion correction in streaming requires to check and turn on the RS2_OPTION_ENABLE_MOTION_CORRECTION option.

lishanggui commented 4 years ago

@ev-mp When I did the above experiments, I confirmed that RS2_OPTION_ENABLE_MOTION_CORRECTION was turned on when running realsense_viewer and rs_motion. Only when I converted by my own program, I turned off RS2_OPTION_ENABLE_MOTION_CORRECTION and then converted. whether you have repeated these experiments, and the results you obtained are What is it？

In the previous stages, I have read the content of your imu calibration program and the content of rs_motion program.

ev-mp commented 4 years ago

@lishanggui, what is the current status on this? Please update

RealSenseSupport commented 4 years ago

Hi @lishanggui

The latest update to D435i IMU calibration came in this PR https://github.com/IntelRealSense/librealsense/pull/6029

Is there any status on this?

*If we don’t hear from you in 7 days, this issue will be closed.

lishanggui commented 4 years ago

@ev-mp @RealSenseSupport In the new version, this problem has been resolved. Thank you！

IntelRealSense / librealsense

Problem with D435i imu calibration #5547

Calibration

Observed accelerations