ethz-asl / kalibr

The Kalibr visual-inertial calibration toolbox
Other
4.37k stars 1.4k forks source link

ZR300 optimization failed #136

Open Cezary-Ziemiecki opened 7 years ago

Cezary-Ziemiecki commented 7 years ago

Hi, I am trying to calibrate camera RealSense ZR300 and I've got this runtime error:

 importing libraries
Initializing IMUs:
  Update rate: 200.0
  Accelerometer:
    Noise density: 0.01 
    Noise density (discrete): 0.141421356237 
    Random walk: 0.0002
  Gyroscope:
    Noise density: 0.005
    Noise density (discrete): 0.0707106781187 
    Random walk: 4e-06
Initializing imu rosbag dataset reader:
    Dataset:          bag4.bag
    Topic:            /camera/imu/data_raw
    Number of messages: 16057
Reading IMU data (/camera/imu/data_raw)
  Read 16057 imu readings over 41.2 seconds                          
Initializing calibration target:
  Type: aprilgrid
  Tags: 
    Rows: 3
    Cols: 3
    Size: 0.047 [m]
    Spacing 0.0141 [m]
Initializing camera chain:
Camera chain - cam0:
  Camera model: pinhole
  Focal length: [306.179, 306.929]
  Principal point: [158.523, 122.748]
  Distortion model: radtan
  Distortion coefficients: [0.06696815, -0.11570779, 0.00017277, -0.00054532]
  baseline: no data available
Initializing camera rosbag dataset reader:
    Dataset:          bag4.bag
    Topic:            /camera/color/image_raw
    Number of images: 619
Extracting calibration target corners
  Extracted corners for 502 images (of 619 images)                              

Building the problem
    Spline order: 6
    Pose knots per second: 100
    Do pose motion regularization: False
        xddot translation variance: 1000000.000000
        xddot rotation variance: 100000.000000
    Bias knots per second: 50
    Do bias motion regularization: True
    Blake-Zisserman on reprojection errors -1
    Acceleration Huber width (sigma): -1.000000
    Gyroscope Huber width (sigma): -1.000000
    Do time calibration: False
    Max iterations: 60
    Time offset padding: 0.010000

Estimating imu-camera rotation prior

Initializing a pose spline with 4120 knots (100.000000 knots per second over 41.200000 seconds)
Gravity was intialized to [-3.35176344 -7.36549785 -5.53927307] [m/s^2]
  Orientation prior camera-imu found as: (T_i_c)
[[ 0.99910519 -0.0262021  -0.03320044]
 [ 0.02384797  0.9973024  -0.06942038]
 [ 0.03492984  0.0685665   0.99703488]]
  Gyro bias prior found as: (b_gyro)
[-0.18099904  0.17929133 -0.07491421]

Initializing a pose spline with 4124 knots (100.000000 knots per second over 41.240000 seconds)

Initializing the bias splines with 2062 knots

Adding camera error terms (/camera/color/image_raw)
  Added 502 camera error terms                               

Adding accelerometer error terms (/camera/imu/data_raw)
  Added 16030 of 16057 accelerometer error terms (skipped 27 out-of-bounds measurements)

Adding gyroscope error terms (/camera/imu/data_raw)
  Added 16030 of 16057 gyroscope error terms (skipped 27 out-of-bounds measurements)

Before Optimization
===================
Normalized Residuals
----------------------------
Reprojection error (cam0):     mean 9.79523014249e+13, median 15.0984437924, std: 1.17843834401e+16
Gyroscope error (imu0):        mean 56.03072579, median 31.1942861176, std: 70.9659426477
Accelerometer error (imu0):    mean 6651.64078185, median 80.5633417412, std: 73526.7088825

Residuals
----------------------------
Reprojection error (cam0) [px]:     mean 9.79523014249e+13, median 15.0984437924, std: 1.17843834401e+16
Gyroscope error (imu0) [rad/s]:     mean 3.96197061609, median 2.2057691248, std: 5.01804992795
Accelerometer error (imu0) [m/s^2]: mean 940.684060573, median 11.393377052, std: 10398.2468898

Optimizing...
Using the block_cholesky linear system solver
Using the levenberg_marquardt trust region policy
Using the block_cholesky linear system solver
Using the levenberg_marquardt trust region policy
Initializing
Optimization problem initialized with 8266 design variables and 46581 error terms
The Jacobian matrix is 125220 x 37184
[0.0]: J: 2.01642e+36
CHOLMOD warning: matrix not positive definite
[WARNING] System solution failed
CHOLMOD warning: matrix not positive definite
[WARNING] System solution failed
CHOLMOD warning: matrix not positive definite
[WARNING] System solution failed
CHOLMOD warning: matrix not positive definite
[WARNING] System solution failed
CHOLMOD warning: matrix not positive definite
[WARNING] System solution failed
CHOLMOD warning: matrix not positive definite
[WARNING] System solution failed
CHOLMOD warning: matrix not positive definite
[WARNING] System solution failed
CHOLMOD warning: matrix not positive definite
[WARNING] System solution failed
CHOLMOD warning: matrix not positive definite
[WARNING] System solution failed
[1]: J: 3.25656e+35, dJ: 1.69076e+36, deltaX: 7.51981e-05, LM - lambda:1.80144e+17 mu:1024
[2]: J: 5.25941e+34, dJ: 2.73062e+35, deltaX: 6.85327e-05, LM - lambda:1.24249e+17 mu:2
[3]: J: 8.49402e+33, dJ: 4.41001e+34, deltaX: 6.0358e-05, LM - lambda:8.56962e+16 mu:2
[4]: J: 1.37179e+33, dJ: 7.12223e+33, deltaX: 6.42506e-05, LM - lambda:5.9106e+16 mu:2
[5]: J: 2.21541e+32, dJ: 1.15024e+33, deltaX: 7.37394e-05, LM - lambda:4.07661e+16 mu:2
[6]: J: 3.57776e+31, dJ: 1.85763e+32, deltaX: 7.84115e-05, LM - lambda:2.81166e+16 mu:2
[7]: J: 5.7777e+30, dJ: 2.99999e+31, deltaX: 8.03781e-05, LM - lambda:1.93919e+16 mu:2
[8]: J: 9.33011e+29, dJ: 4.84469e+30, deltaX: 0.000115765, LM - lambda:1.33741e+16 mu:2
[9]: J: 1.50684e+29, dJ: 7.82327e+29, deltaX: 0.000152262, LM - lambda:9.22326e+15 mu:2
[10]: J: 2.43609e+28, dJ: 1.26324e+29, deltaX: 0.000172253, LM - lambda:6.36002e+15 mu:2
[11]: J: 3.96325e+27, dJ: 2.03976e+28, deltaX: 0.000144377, LM - lambda:4.38462e+15 mu:2
[12]: J: 6.66817e+26, dJ: 3.29643e+27, deltaX: 0.000234175, LM - lambda:3.02106e+15 mu:2
[13]: J: 1.30051e+26, dJ: 5.36765e+26, deltaX: 0.000361048, LM - lambda:2.07623e+15 mu:2
[14]: J: 3.91909e+25, dJ: 9.08605e+25, deltaX: 0.000516712, LM - lambda:1.40464e+15 mu:2
[15]: J: 2.118e+25, dJ: 1.80109e+25, deltaX: 0.000755783, LM - lambda:8.70833e+14 mu:2
[16]: J: 1.48544e+25, dJ: 6.32564e+24, deltaX: 0.00190871, LM - lambda:3.07704e+14 mu:2
[17]: J: 9.39613e+24, dJ: 5.45826e+24, deltaX: 0.00580215, LM - lambda:1.02568e+14 mu:2
[18]: J: 3.13532e+24, dJ: 6.26081e+24, deltaX: 0.019304, LM - lambda:3.41894e+13 mu:2
[19]: J: 7.77287e+23, dJ: 2.35804e+24, deltaX: 0.0311133, LM - lambda:1.13965e+13 mu:2
[20]: J: 3.87619e+23, dJ: 3.89668e+23, deltaX: 0.0306516, LM - lambda:3.79882e+12 mu:2
[21]: J: 1.51024e+23, dJ: 2.36595e+23, deltaX: 0.0174174, LM - lambda:3.77738e+12 mu:2
[22]: J: 9.42155e+22, dJ: 5.68083e+22, deltaX: 0.0251883, LM - lambda:2.8747e+12 mu:2
[23]: J: 5.75907e+22, dJ: 3.66248e+22, deltaX: 0.0343251, LM - lambda:2.07534e+12 mu:2
[24]: J: 2.14216e+22, dJ: 3.6169e+22, deltaX: 0.109791, LM - lambda:6.91779e+11 mu:2
[25]: J: 5.27504e+21, dJ: 1.61466e+22, deltaX: 0.145624, LM - lambda:2.30593e+11 mu:2
[26]: J: 1.0581e+21, dJ: 4.21695e+21, deltaX: 0.110433, LM - lambda:1.3985e+11 mu:2
[27]: J: 1.91624e+20, dJ: 8.66473e+20, deltaX: 0.130909, LM - lambda:5.84358e+10 mu:2
[28]: J: 3.10066e+19, dJ: 1.60617e+20, deltaX: 0.124984, LM - lambda:2.11688e+10 mu:2
[29]: J: 5.10414e+18, dJ: 2.59025e+19, deltaX: 0.103959, LM - lambda:9.88639e+09 mu:2
[30]: J: 8.95677e+17, dJ: 4.20846e+18, deltaX: 0.0889972, LM - lambda:4.59992e+09 mu:2
[31]: J: 1.7261e+17, dJ: 7.23067e+17, deltaX: 0.0740166, LM - lambda:2.20851e+09 mu:2
[32]: J: 3.36551e+16, dJ: 1.38955e+17, deltaX: 0.0623273, LM - lambda:1.07661e+09 mu:2
[33]: J: 6.50338e+15, dJ: 2.71518e+16, deltaX: 0.0536605, LM - lambda:4.81653e+08 mu:2
[34]: J: 1.65793e+15, dJ: 4.84544e+15, deltaX: 0.0432395, LM - lambda:2.16559e+08 mu:2
[35]: J: 7.82287e+14, dJ: 8.75647e+14, deltaX: 0.0346319, LM - lambda:1.06312e+08 mu:2
[36]: J: 6.26027e+14, dJ: 1.56261e+14, deltaX: 0.049082, LM - lambda:6.60185e+07 mu:2
Last step was a regression. Reverting
[37]: J: 3.64923e+16, dJ: -3.58662e+16, deltaX: 0.0300082, LM - lambda:6.56398e+07 mu:2
[38]: J: 5.81653e+14, dJ: 4.43736e+13, deltaX: 0.00485007, LM - lambda:2.62559e+08 mu:4
[39]: J: 5.64976e+14, dJ: 1.66771e+13, deltaX: 0.0021627, LM - lambda:2.67685e+08 mu:2
Last step was a regression. Reverting
[40]: J: 5.66401e+14, dJ: -1.4245e+12, deltaX: 0.00204849, LM - lambda:2.99256e+08 mu:2
[41]: J: 5.63551e+14, dJ: 1.42522e+12, deltaX: 0.000717426, LM - lambda:1.19702e+09 mu:4
[42]: J: 5.59169e+14, dJ: 4.38191e+12, deltaX: 0.000526942, LM - lambda:2.16751e+09 mu:2
[43]: J: 5.55221e+14, dJ: 3.94751e+12, deltaX: 0.000278583, LM - lambda:3.15468e+09 mu:2
Last step was a regression. Reverting
[44]: J: 5.55963e+14, dJ: -7.41628e+11, deltaX: 0.000167361, LM - lambda:4.40261e+09 mu:2
[45]: J: 5.45253e+14, dJ: 9.96837e+12, deltaX: 0.000119745, LM - lambda:1.76104e+10 mu:4
[46]: J: 5.43927e+14, dJ: 1.32626e+12, deltaX: 4.83525e-05, LM - lambda:1.69808e+10 mu:2
[47]: J: 5.4359e+14, dJ: 3.36735e+11, deltaX: 1.69349e-05, LM - lambda:1.64675e+10 mu:2
[48]: J: 5.43425e+14, dJ: 1.65002e+11, deltaX: 1.30692e-05, LM - lambda:1.32282e+10 mu:2
[49]: J: 5.43235e+14, dJ: 1.89958e+11, deltaX: 3.87857e-05, LM - lambda:4.40941e+09 mu:2
[50]: J: 5.43209e+14, dJ: 2.60994e+10, deltaX: 1.51802e-05, LM - lambda:3.8865e+09 mu:2
[51]: J: 5.43191e+14, dJ: 1.77885e+10, deltaX: 1.49872e-05, LM - lambda:4.04593e+09 mu:2
[52]: J: 5.43182e+14, dJ: 8.85602e+09, deltaX: 1.17357e-05, LM - lambda:4.9286e+09 mu:2
[53]: J: 5.43173e+14, dJ: 9.37377e+09, deltaX: 1.26221e-05, LM - lambda:7.72498e+09 mu:2
[54]: J: 5.43145e+14, dJ: 2.75419e+10, deltaX: 1.08178e-05, LM - lambda:1.21233e+10 mu:2
[55]: J: 5.43129e+14, dJ: 1.68309e+10, deltaX: 7.64468e-06, LM - lambda:1.28254e+10 mu:2
[56]: J: 5.43119e+14, dJ: 9.35821e+09, deltaX: 5.12794e-06, LM - lambda:1.30747e+10 mu:2
[57]: J: 5.43114e+14, dJ: 5.46634e+09, deltaX: 3.42053e-06, LM - lambda:1.31112e+10 mu:2
[58]: J: 5.4311e+14, dJ: 3.75373e+09, deltaX: 2.24482e-06, LM - lambda:1.31072e+10 mu:2
[59]: J: 5.43107e+14, dJ: 3.39583e+09, deltaX: 1.67594e-06, LM - lambda:1.19027e+10 mu:2
[60]: J: 5.43081e+14, dJ: 2.57645e+10, deltaX: 1.47878e-05, LM - lambda:3.96755e+09 mu:2
[ERROR] [1508215661.949494]: Optimization failed!
Traceback (most recent call last):
  File "/root/kalibr_workspace/devel/bin/kalibr_calibrate_imu_camera", line 6, in <module>
    exec(fh.read())
  File "<string>", line 236, in <module>
  File "<string>", line 206, in main
  File "/root/kalibr_workspace/src/Kalibr/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccCalibrator.py", line 179, in optimize
    raise RuntimeError("Optimization failed!")
RuntimeError: Optimization failed!

For sure, I set max_iter to high value (60000) and left my office. Today's morning it was still calibrate zrzut ekranu z 2017-10-17 06-25-38 . Do you have an idea what could cause a calibration problem? It could be wrong bag, Apriltag target or cam/imu_config. By the way, Did someone calibrate this camera and have the appropriate data to create a config file? I think that bad transform data between imu and camera is a main reason.

CtfChan commented 6 years ago

Hey I'm having the same problem. Did you figure out where your issue was?

Mcthomas777 commented 3 years ago

Hey, I'm facing these issues too...Did you guys found something out there, I checked my IMU samples (good units, good looking), pictures too... so I don't know what to do

ZhihaoZhang0618 commented 2 years ago

Please help us. So many wrongs. :(