Questions about calibration results

[nikisim]

Hi!

I'm not sure that my calibtation results is good enough. Rotation from imu to camera seems pretty well for me, and translation between imu and cam is about 19 cm (in real life it's about 21-22 cm).

I guess, accelerometer bias and gyro bias plots are wrong. Also I'm not sure about reprojection errors of camera...

I'm using PixHawk 2.6.8 and simple Logitech web-camera (I'm just testing the calibration with that configuration)

Calibration results
===================
Normalized Residuals
----------------------------
Reprojection error (cam0):     mean 1.1150435780949453, median 0.5946703300528435, std: 1.8088305216956Calibration results
===================
Normalized Residuals
----------------------------
Reprojection error (cam0):     mean 1.1150435780949453, median 0.5946703300528435, std: 1.808830521695631
Gyroscope error (imu0):        mean 7.520513566778695, median 5.778769319718048, std: 7.625053741693555
Accelerometer error (imu0):    mean 1.2962267561460572, median 0.8374503441834276, std: 1.4585924343037173

Residuals
----------------------------
Reprojection error (cam0) [px]:     mean 1.1150435780949453, median 0.5946703300528435, std: 1.808830521695631
Gyroscope error (imu0) [rad/s]:     mean 0.014024295390039682, median 0.010776281062589484, std: 0.014219242487749709
Accelerometer error (imu0) [m/s^2]: mean 0.06477636764077423, median 0.04184992411127651, std: 0.07289027117710822

Transformation (cam0):
-----------------------
T_ci:  (imu0 to cam0): 
[[ 0.01067668 -0.99820924  0.05885844  0.01736561]
 [-0.15374868 -0.05980062 -0.98629875  0.01852371]
 [ 0.9880523   0.00148099 -0.15411183 -0.19061944]
 [ 0.          0.          0.          1.        ]]

T_ic:  (cam0 to imu0): 
[[ 0.01067668 -0.15374868  0.9880523   0.19100456]
 [-0.99820924 -0.05980062  0.00148099  0.01872455]
 [ 0.05885844 -0.98629875 -0.15411183 -0.01212891]
 [ 0.          0.          0.          1.        ]]

timeshift cam0 to imu0: [s] (t_imu = t_cam + shift)
-0.04178168841504161

Gravity vector in target coords: [m/s^2]
[-0.44292785 -9.79471593  0.18914968]

Calibration configuration
=========================

cam0
-----
  Camera model: pinhole
  Focal length: [644.2606782785358, 646.1421152393555]
  Principal point: [331.2809204759637, 247.85828379347475]
  Distortion model: equidistant
  Distortion coefficients: [0.35643839608777994, 0.47947760274633366, -3.8642155892653363, 8.300856583097026]
  Type: aprilgrid
  Tags: 
    Rows: 6
    Cols: 6
    Size: 0.021 [m]
    Spacing 0.005999994000000001 [m]

IMU configuration
=================

IMU0:
 ----------------------------
  Model: calibrated
  Update rate: 200.0
  Accelerometer:
    Noise density: 0.0035336262426497226 
    Noise density (discrete): 0.04997302156712719 
    Random walk: 0.0002959774687388282
  Gyroscope:
    Noise density: 0.00013186166454730511
    Noise density (discrete): 0.0018648055435989041 
    Random walk: 1.3210249901572838e-06
  T_ib (imu0 to imu0)
    [[1. 0. 0. 0.]
     [0. 1. 0. 0.]
     [0. 0. 1. 0.]
     [0. 0. 0. 1.]]
  time offset with respect to IMU0: 0.0 [s]31
Gyroscope error (imu0):        mean 7.520513566778695, median 5.778769319718048, std: 7.625053741693555
Accelerometer error (imu0):    mean 1.2962267561460572, median 0.8374503441834276, std: 1.4585924343037173

Residuals
----------------------------
Reprojection error (cam0) [px]:     mean 1.1150435780949453, median 0.5946703300528435, std: 1.808830521695631
Gyroscope error (imu0) [rad/s]:     mean 0.014024295390039682, median 0.010776281062589484, std: 0.014219242487749709
Accelerometer error (imu0) [m/s^2]: mean 0.06477636764077423, median 0.04184992411127651, std: 0.07289027117710822

Transformation (cam0):
-----------------------
T_ci:  (imu0 to cam0): 
[[ 0.01067668 -0.99820924  0.05885844  0.01736561]
 [-0.15374868 -0.05980062 -0.98629875  0.01852371]
 [ 0.9880523   0.00148099 -0.15411183 -0.19061944]
 [ 0.          0.          0.          1.        ]]

T_ic:  (cam0 to imu0): 
[[ 0.01067668 -0.15374868  0.9880523   0.19100456]
 [-0.99820924 -0.05980062  0.00148099  0.01872455]
 [ 0.05885844 -0.98629875 -0.15411183 -0.01212891]
 [ 0.          0.          0.          1.        ]]

timeshift cam0 to imu0: [s] (t_imu = t_cam + shift)
-0.04178168841504161

Gravity vector in target coords: [m/s^2]
[-0.44292785 -9.79471593  0.18914968]

Calibration configuration
=========================

cam0
-----
  Camera model: pinhole
  Focal length: [644.2606782785358, 646.1421152393555]
  Principal point: [331.2809204759637, 247.85828379347475]
  Distortion model: equidistant
  Distortion coefficients: [0.35643839608777994, 0.47947760274633366, -3.8642155892653363, 8.300856583097026]
  Type: aprilgrid
  Tags: 
    Rows: 6
    Cols: 6
    Size: 0.021 [m]
    Spacing 0.005999994000000001 [m]

IMU configuration
=================

IMU0:
 ----------------------------
  Model: calibrated
  Update rate: 200.0
  Accelerometer:
    Noise density: 0.0035336262426497226 
    Noise density (discrete): 0.04997302156712719 
    Random walk: 0.0002959774687388282
  Gyroscope:
    Noise density: 0.00013186166454730511
    Noise density (discrete): 0.0018648055435989041 
    Random walk: 1.3210249901572838e-06

Can anyone explain me, what's wrong with calibration results? I would be thankful for any answer.

Thanks

[goldbattle]

It looks like there are some issues around 25-75 seconds in. Maybe you could process the bag just after these? The IMU also seems to have some timing problems of either dropped messages or just irregular publishing (those 10ms spikes). Your reprojection has very large errors and are non-gaussian suggesting something wrong with the selected model, camera, or dataset. One thought is if it is a webcam, it might be a rolling shutter camera which will cause issues with distortion of the collected image (Kalibr IMU-camera calibration assumes global shutter cameras).

[nikisim]

Thank you very much for your answer!

1) Problems between 25-75 seconds. I kinda don't understand how should "Comparsion of predicted and measured specific force" plot looks like. I get how to validate other plots: if blue lines inside red bounds - it's okay :) But these est. bodyspline - I don't know how to interpretate

2) Yeah, I know that IMU hz plot should be just one line without any spread and other lines. But how can I achive this? As I understood, I need to rectify the timestamp problems, but how exactly - I have no idea. Although with IMU PixHawk this plot looks much better than IMU from A1 Unitree robot... You can see the last one below. (both IMU was calibrated with allan-variance-ros)

Sorry, if I ask some dumb questions...

[nikisim]

One more question: is it possible to use Kalibr for imu->cam calibration with IMU 10 Hz? Or it is not going to work in this case?

[goldbattle]

A 10Hz IMU probably will not work. You should use a higher frequency IMU. You would need to look into th specific IMU driver you are using to see how to fix the timestamp problems. The posted IMU rate for the A1 looks very very bad so you should definitely take the time to look at that driver.