Calibration of Cameras and IMU.

[Hateley123]

Everytime I try to calibrate my imu and cameras I get the following output. I am able to use kalibr to calibrate my cameras individually, and my rosbag is fine as I used it to calibrate each camera and get the imu noise. However, anytime I try to calibrate systems together Kalibr keeps throwing exceptions and producing errors. Does anyone know how to get around these. I have copied by yamls below.

Camera Yaml

cam0: #F1
  camera_model: ds
  distortion_coeffs: []
  distortion_model: none
  intrinsics: [-0.2925283388953137, 0.5557001286352555, 373.4591078162464, 372.3986581382408, 639.4945895431914, 473.6470362461054]

  T_cam_imu:
  - [0.0101, 0.0060, -1, -0.1863]
  - [-1, 0.0014, -0.0101, -0.0022]
  - [-0.0003, 1.0058, 0.0416, 0.0143]
  - [0.0, 0.0, 0.0, 1.0]
  timeshift_cam_imu: -1e-5
  resolution: [1280, 1024]
  rostopic: /F1/image_raw
cam1: #F2  
  camera_model: ds
  distortion_coeffs: []
  distortion_model: none
  intrinsics: [-0.15748417798406494, 0.607481973424445, 467.3988450031068, 466.8662726102595, 634.1810050580968, 496.0138669711759]
  T_cn_cnm1:
  - [0.994, 0.0062, -0.0354, -0.1527122]
  - [-0.0062, 1, 0.0015, 0.0015924]
  - [0.0354, -0.0013, 0.994, -0.0067475]
  - [0.0, 0.0, 0.0, 1.0]
  T_cam_imu:
  - [0.0101, 0.0060, -1, -0.0336]
  - [-1, 0.0014, -0.0101, -0.0038]
  - [-0.0003, 1.0058, 0.0416, 0.0224]
  - [0.0, 0.0, 0.0, 1.0]

  time_shift_cam_imu: -1e-5
  resolution: [1280, 1024]
  rostopic: /F2/image_raw

IMU Yaml

#Accelerometers
accelerometer_noise_density: 0.000254 # white noise density
accelerometer_random_walk: 0.000095 # random walk

gyroscope_noise_density: 0.001283 #white noise
gyroscope_random_walk: 0.11805 # random walk

rostopic: /imu/data 
update_rate: 200.0

System Output

importing libraries
Initializing IMUs:
  Update rate: 200.0
  Accelerometer:
    Noise density: 0.000254 
    Noise density (discrete): 0.0035921024484276616 
    Random walk: 9.5e-05
  Gyroscope:
    Noise density: 0.001283
    Noise density (discrete): 0.01814436000524681 
    Random walk: 0.11805
Initializing imu rosbag dataset reader:
    Dataset:          /home/ig-handle/2023_01_19_11_15_42/cam_imu.bag
    Topic:            /imu/data
    Number of messages: 6257
Reading IMU data (/imu/data)
  Read 6257 imu readings over 31.3 seconds                        
Initializing calibration target:
  Type: checkerboard
  Rows
    Count: 6
    Distance: 0.081 [m]
  Cols
    Count: 7
    Distance: 0.081 [m]
Initializing camera chain:
Camera chain - cam0:
  Camera model: ds
  Focal length: [373.4591078162464, 372.3986581382408]
  Principal point: [639.4945895431914, 473.6470362461054]
  DS xi: -0.2925283388953137
  DS alpha: 0.5557001286352555
  Distortion model: none
  Distortion coefficients: []
  baseline: no data available
Camera chain - cam1:
  Camera model: ds
  Focal length: [467.3988450031068, 466.8662726102595]
  Principal point: [634.1810050580968, 496.0138669711759]
  DS xi: -0.15748417798406494
  DS alpha: 0.607481973424445
  Distortion model: none
  Distortion coefficients: []
  baseline: no data available
Initializing camera rosbag dataset reader:
    Dataset:          /home/ig-handle/2023_01_19_11_15_42/cam_imu.bag
    Topic:            /F1/image_raw
    Number of images: 623
Extracting calibration target corners
  Extracted corners for 206 images (of 623 images)                              
Initializing camera rosbag dataset reader:
    Dataset:          /home/ig-handle/2023_01_19_11_15_42/cam_imu.bag
    Topic:            /F2/image_raw
    Number of images: 623
Extracting calibration target corners
  Extracted corners for 228 images (of 623 images)                              
Traceback (most recent call last):
  File "/home/ig-handle/catkin_ws/src/kalibr_workspace/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_common/ConfigReader.py", line 697, in getExtrinsicsLastCamToHere
    t = trafo.T() #for error checking
RuntimeError: [std::runtime_error] /home/ig-handle/catkin_ws/src/kalibr_workspace/src/kalibr/Schweizer-Messer/sm_kinematics/src/quaternion_algebra.cpp:79: quat2r() debug assert(q.norm() == 1.f) failed [0.998661 == 1 (0.00133871 > 0.0001)]: The quaternion must be a unit vector to represent a rotation

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ig-handle/catkin_ws/devel/lib/kalibr/kalibr_calibrate_imu_camera", line 15, in <module>
    exec(compile(fh.read(), python_script, 'exec'), context)
  File "/home/ig-handle/catkin_ws/src/kalibr_workspace/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_calibrate_imu_camera", line 247, in <module>
    main()
  File "/home/ig-handle/catkin_ws/src/kalibr_workspace/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_calibrate_imu_camera", line 175, in main
    camChain = sens.IccCameraChain(chain, targetConfig, parsed)
  File "/home/ig-handle/catkin_ws/src/kalibr_workspace/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccSensors.py", line 441, in __init__
    self.initializeBaselines()
  File "/home/ig-handle/catkin_ws/src/kalibr_workspace/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccSensors.py", line 447, in initializeBaselines
    self.camList[camNr].T_extrinsic = self.chainConfig.getExtrinsicsLastCamToHere(camNr)
  File "/home/ig-handle/catkin_ws/src/kalibr_workspace/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_common/ConfigReader.py", line 187, in func
    return f(*args, **kwargs)
  File "/home/ig-handle/catkin_ws/src/kalibr_workspace/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_common/ConfigReader.py", line 699, in getExtrinsicsLastCamToHere
    self.raiseError("invalid camera baseline (cam{0} in {1})".format(camNr, self.yamlFile))
  File "/home/ig-handle/catkin_ws/src/kalibr_workspace/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_common/ConfigReader.py", line 235, in raiseError
    raise RuntimeError( "{0}{1}".format(header, message) )
RuntimeError: [CameraChainParameters Reader]: invalid camera baseline (cam1 in /home/ig-handle/catkin_ws/src/ig_handle/cam_stero.yaml)

[goldbattle]

It looks like you have inputted an invalid rotation matrix, thus it is complaining about the magnitude of the quaterion.

quat2r() debug assert(q.norm() == 1.f) failed [0.998661 == 1 (0.00133871 > 0.0001)]: The quaternion must be a unit vector to represent a rotation

[Hateley123]

Hi ya we got it to work, the target was fine apparently there was an issue with our time stamps. Even though both cameras activated at same time the time stamps for the images were wildly different. once we fixed the timing it worked.