search

ethz-asl / kalibr

The Kalibr visual-inertial calibration toolbox
Other
4.44k stars 1.4k forks source link

IMU / camera calibration failure #637

Closed lsorgi closed 1 year ago

lsorgi commented 1 year ago

calibrating divergent stereo system + IMU

stereo calibration works nicely, all metrics are good IMU calibration fails

below the calibration log any hint? thanks!!

Initializing IMUs:
  Update rate: 200
  Accelerometer:
    Noise density: 0.005116692151038151
    Noise density (discrete): 0.07236095434486117
    Random walk: 0.00026474031165016874
  Gyroscope:
    Noise density: 0.00011493814460992904
    Noise density (discrete): 0.001625470829413617
    Random walk: 1.247322870181146e-06
Initializing imu rosbag dataset reader:
        Dataset:          /data/camera-calibrations/e45f0149f866/session8_sync_fix/kalibr.bag
        Topic:            /imu
        Number of messages: 10111
Reading IMU data (/imu)
  Read 10111 imu readings over 57.0 seconds
Initializing calibration target:
  Type: aprilgrid
  Tags:
    Rows: 5
    Cols: 8
    Size: 0.123 [m]
    Spacing 0.03075 [m]
Initializing camera chain:
Camera chain - cam0:
  Camera model: pinhole
  Focal length: [330.28704759624384, 330.01092469759624]
  Principal point: [302.7617466020397, 199.71728642440976]
  Distortion model: equidistant
  Distortion coefficients: [0.13182355980648416, -0.03897622526636737, -0.008526960864471788, 0.00848591125342941]
  baseline: no data available
Camera chain - cam1:
  Camera model: pinhole
  Focal length: [308.5013056069892, 308.3387367432519]
  Principal point: [292.8905330635243, 198.0194029743133]
  Distortion model: equidistant
  Distortion coefficients: [0.001316903130525178, -0.007774504378675862, 0.00826606811162802, -0.0025003284493744955]
  baseline: [[ 0.7073834  -0.01147902  0.70673684  0.11204606]
 [ 0.02103858  0.99976705 -0.00481932 -0.00048946]
 [-0.70651688  0.01827785  0.70746012 -0.05227686]
 [ 0.          0.          0.          1.        ]]
Initializing camera rosbag dataset reader:
        Dataset:          /data/camera-calibrations/e45f0149f866/session8_sync_fix/kalibr.bag
        Topic:            /cam0/image_raw
        Number of images: 1100
Extracting calibration target corners
  Extracted corners for 788 images (of 1100 images)
Initializing camera rosbag dataset reader:
        Dataset:          /data/camera-calibrations/e45f0149f866/session8_sync_fix/kalibr.bag
        Topic:            /cam1/image_raw
        Number of images: 1095
Extracting calibration target corners
  Extracted corners for 920 images (of 1095 images)
Baseline between cam0 and cam1 set to:
T=  [[ 0.7073834  -0.01147902  0.70673684  0.11204606]
 [ 0.02103858  0.99976705 -0.00481932 -0.00048946]
 [-0.70651688  0.01827785  0.70746012 -0.05227686]
 [ 0.          0.          0.          1.        ]]
Baseline:  0.12364234936168329  [m]

Building the problem
        Spline order: 6
        Pose knots per second: 100
        Do pose motion regularization: False
                xddot translation variance: 1000000.000000
                xddot rotation variance: 100000.000000
        Bias knots per second: 50
        Do bias motion regularization: True
        Blake-Zisserman on reprojection errors -1
        Acceleration Huber width (sigma): -1.000000
        Gyroscope Huber width (sigma): -1.000000
        Do time calibration: True
        Max iterations: 30
        Time offset padding: 0.030000
Estimating time shift camera to imu:

Initializing a pose spline with 5454 knots (100.000000 knots per second over 54.536747 seconds)
  Time shift camera to imu (t_imu = t_cam + shift):
-9.922897582591489
Estimating time shift camera to imu:

Initializing a pose spline with 5449 knots (100.000000 knots per second over 54.486757 seconds)
  Time shift camera to imu (t_imu = t_cam + shift):
-9.922897582591489

Estimating imu-camera rotation prior

Initializing a pose spline with 5454 knots (100.000000 knots per second over 54.536747 seconds)
Gravity was intialized to [-1.86802715  7.73625604 -5.72968061] [m/s^2]
  Orientation prior camera-imu found as: (T_i_c)
[[-0.25163625  0.61439158  0.74779822]
 [-0.12423145  0.74576411 -0.65452459]
 [-0.95981547 -0.25760217 -0.11133453]]
  Gyro bias prior found as: (b_gyro)
[ 0.02523795  0.01354615 -0.02418514]

Initializing a pose spline with 5466 knots (100.000000 knots per second over 54.656747 seconds)

Initializing the bias splines with 2733 knots

Adding camera error terms (/cam0/image_raw)
  Added 788 camera error terms

Adding camera error terms (/cam1/image_raw)
  Added 920 camera error terms

Adding accelerometer error terms (/imu)
  Added 9696 of 10111 accelerometer error terms (skipped 415 out-of-bounds measurements)

Adding gyroscope error terms (/imu)
  Added 9696 of 10111 gyroscope error terms (skipped 415 out-of-bounds measurements)

Before Optimization
===================
Normalized Residuals
----------------------------
Reprojection error (cam0):     mean 0.920738979418543, median 0.7957606562259253, std: 0.757707247808881
Reprojection error (cam1):     mean 6.979708357069321, median 0.8753592697236781, std: 17.94009898423782
Gyroscope error (imu0):        mean 386.2875035064701, median 282.7314144895927, std: 292.0475888836684
Accelerometer error (imu0):    mean 52.298186329495614, median 46.61385703693466, std: 28.91990045293523

Residuals
----------------------------
Reprojection error (cam0) [px]:     mean 0.920738979418543, median 0.7957606562259253, std: 0.757707247808881
Reprojection error (cam1) [px]:     mean 6.979708357069321, median 0.8753592697236781, std: 17.94009898423782
Gyroscope error (imu0) [rad/s]:     mean 0.6278990687167775, median 0.4595716668116834, std: 0.47471483653098356
Accelerometer error (imu0) [m/s^2]: mean 3.7843466733076756, median 3.3730231808875146, std: 2.0926715963327767

Optimizing...
Using the block_cholesky linear system solver
Using the levenberg_marquardt trust region policy
Using the block_cholesky linear system solver
Using the levenberg_marquardt trust region policy
Initializing
Optimization problem initialized with 10952 design variables and 186146 error terms
The Jacobian matrix is 391682 x 49264
[0.0]: J: 2.34456e+09
Exception in thread block: Exception in thread block: [aslam::Exception] /catkin_ws/src/kalibr/aslam_nonparametric_estimation/aslam_splines/src/BSplineExpressions.cpp:447: toTransformationMatrixImplementation() assert(_bufferTmin <= _time.toScalar() < _bufferTmax) failed [454.36 <= 454.321 < 454.43]: Spline Coefficient Buffer Exceeded. Set larger buffer margins![aslam::Exception] /catkin_ws/src/kalibr/aslam_nonparametric_estimation/aslam_splines/src/BSplineExpressions.cpp:447: toTransformationMatrixImplementation() assert(_bufferTmin <= _time.toScalar() < _bufferTmax) failed [467.059 <= 467.019 < 467.129]: Spline Coefficient Buffer Exceeded. Set larger buffer margins!Exception in thread block: Exception in thread block: [aslam::Exception] /catkin_ws/src/kalibr/aslam_nonparametric_estimation/aslam_splines/src/BSplineExpressions.cpp:447: toTransformationMatrixImplementation() assert(_bufferTmin <= _time.toScalar() < _bufferTmax) failed [427.011 <= 426.978 < 427.081]: Spline Coefficient Buffer Exceeded. Set larger buffer margins!
[aslam::Exception] /catkin_ws/src/kalibr/aslam_nonparametric_estimation/aslam_splines/src/BSplineExpressions.cpp:447: toTransformationMatrixImplementation() assert(_bufferTmin <= _time.toScalar() < _bufferTmax) failed [471.759 <= 471.717 < 471.829]: Spline Coefficient Buffer Exceeded. Set larger buffer margins!

Exception in thread block: [aslam::Exception] /catkin_ws/src/kalibr/aslam_nonparametric_estimation/aslam_splines/src/BSplineExpressions.cpp:447: toTransformationMatrixImplementation() assert(_bufferTmin <= _time.toScalar() < _bufferTmax) failed [439.31 <= 439.275 < 439.38]: Spline Coefficient Buffer Exceeded. Set larger buffer margins!
Exception in thread block: [aslam::Exception] /catkin_ws/src/kalibr/aslam_nonparametric_estimation/aslam_splines/src/BSplineExpressions.cpp:447: toTransformationMatrixImplementation() assert(_bufferTmin <= _time.toScalar() < _bufferTmax) failed [455.81 <= 455.772 < 455.88]: Spline Coefficient Buffer Exceeded. Set larger buffer margins!
Exception in thread block:
[aslam::Exception] /catkin_ws/src/kalibr/aslam_nonparametric_estimation/aslam_splines/src/BSplineExpressions.cpp:447: toTransformationMatrixImplementation() assert(_bufferTmin <= _time.toScalar() < _bufferTmax) failed [422.771 <= 422.73 < 422.841]: Spline Coefficient Buffer Exceeded. Set larger buffer margins!
[ERROR] [1689881784.002648]: std::exception
[ERROR] [1689881784.005585]: Optimization failed!
Traceback (most recent call last):
  File "/catkin_ws/devel/lib/kalibr/kalibr_calibrate_imu_camera", line 15, in <module>
    exec(compile(fh.read(), python_script, 'exec'), context)
  File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_calibrate_imu_camera", line 247, in <module>
    main()
  File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_calibrate_imu_camera", line 209, in main
    iCal.optimize(maxIterations=parsed.max_iter, recoverCov=parsed.recover_cov)
  File "/catkin_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccCalibrator.py", line 180, in optimize
    raise RuntimeError("Optimization failed!")
RuntimeError: Optimization failed!
goldbattle commented 1 year ago

Likely you have bad timestamps or some other hardware problem. I recommend inspecting these first.