Iteration time in spline fusion

[fickrie67]

Hello, thanks for the great repo. I attempted to perform the camera to IMU calibration using the provided GoPro9 dataset. However, it has taken up to 44 iterations so far and still going. Is this duration considered normal?

[urbste]

Hi, no this is not really normal. Usually it should converge after 5-10 iterations. Could you please post the output of the calibration?

[fickrie67]

Thank you for your response. Eventually, it stopped after 50 iterations. When I attempted to run it with my dataset, it did the same behavior, running iterations up to 50 times, which takes a bit time..here is my result from my dataset: log_openicc.txt

[fickrie67]

I tried to run your dataset again. and it is finally able to converge after 40 iterations..but not 5-10 iterations though..here is the result log_openicc1.txt

[fickrie67]

one more thing, after i have done the calibration, where can i find the transformation from camera to imu matrix? hence it is 4x4 matrix..

[urbste]

I could not reproduce your issues. However in the same instances I just pushed a major update. Please have a look at the Readme. In addition, I verified that the calibration works as expected. Here is the output of a calibration in the docker environment:

successfull_calib.txt

Could you please update your installation and build and check again if it works?

[urbste]

I just also verified with my local build that it works: (base) steffen@steffen-IdeaPad-5-Pro-16ARH7:~/projects/OpenImuCameraCalibrator$ python python/run_gopro_calibration.py --path_calib_dataset /home/steffen/Downloads/GoPro9/dataset3/ --path_to_build ./build/applications/

Also here it works as expected:

Termination: CONVERGENCE (Function tolerance reached. |cost_change|/cost: 8.272755e-05 <= 1.000000e-04)

Mean reprojection error 1.00979 number residuals: 227334 I0108 18:02:28.883790 269841 continuous_time_imu_to_camera_calibration.cc:222] Mean reprojection error 1.00979px I0108 18:02:28.883800 269841 continuous_time_imu_to_camera_calibration.cc:223] Mean reprojection error after line delay optim 1.00979px g: 0.422354 -9.82961 0.127258 accel_bias at time 0: -0.0288142 -0.0365272 0.0100668 gyro_bias at time 0: -0.00160678 0.00187235 -0.00758646 T_i_c qw,qx,qy,qz: 0.00023031 -0.00261572 -0.709489 0.704712 T_i_c t: 0.00322658 -0.0197022 0.00300103 T_i_c R: -0.999986 0.00338705 -0.00401347 0.00403626 0.00674939 -0.999969 -0.00335986 -0.999971 -0.00676297 Initialized line delay [us]: 30.8951 Calibrated line delay [us]: 30.8951

[urbste]

one more thing, after i have done the calibration, where can i find the transformation from camera to imu matrix? hence it is 4x4 matrix..

In two ways. First from the terminal plot. It shows you

T_i_c qw,qx,qy,qz: 0.00023031 -0.00261572 -0.709489 0.704712
T_i_c t: 0.00322658 -0.0197022 0.00300103
T_i_c R: -0.999986 0.00338705 -0.00401347
0.00403626 0.00674939 -0.999969
-0.00335986 -0.999971 -0.00676297

Which is the camera to imu transformation values (R and t and also R as q quaternion). Second, in the cam_imu/ folder there is a file: cam_imu_calibresult*.json that hold these values (t_i_c and q_i_c) and the spline as well:

{
    "calib_line_delay_us": 30.89506172839506,
    "final_reproj_error": 1.0097885513152773,
    "init_line_delay_us": 30.89506172839506,
    "q_i_c": {
        "w": 0.0002303102780159178,
        "x": -0.002615724404776922,
        "y": -0.7094890010055437,
        "z": 0.7047116164752572
    },
    "r3_dt": 0.08977125847059694,
    "so3_dt": 0.05007449488429904,
    "t_i_c": {
        "x": 0.0032265786213766905,
        "y": -0.01970224237895811,
        "z": 0.003001026628285531
    },
    "time_offset_imu_to_cam_s": -0.012422480060567525,
    "trajectory": {
        "10000478554": {
            "accl_bias": {
                "x": -0.028814204520209764,
                "y": -0.036527212881672745,
                "z": 0.010066784027324013
            },
            "accl_imu": {
                "x": 2.2278177458033572,
                "y": 1.5443645083932853,
                "z": 9.18705035971223
            },
            "accl_spline": {
                "x": 2.262389873169382,
                "y": 1.546715779323872,
                "z": 9.224113969697909
            },
            "gyro_bias": {
                "x": -0.0016067765119457745,
                "y": 0.0018723505919155672,
                "z": -0.0075864595615467885
            },
            "gyro_imu": {
                "x": -0.09158679446219382,
                "y": -0.07028753993610223,
                "z": 0.37486687965921195
            },
            "gyro_spline": {
                "x": -0.12909659519304093,
                "y": -0.04729146837371248,
                "z": 0.38385200473285924
            }
        },

[fickrie67]

I just also verified with my local build that it works: (base) steffen@steffen-IdeaPad-5-Pro-16ARH7:~/projects/OpenImuCameraCalibrator$ python python/run_gopro_calibration.py --path_calib_dataset /home/steffen/Downloads/GoPro9/dataset3/ --path_to_build ./build/applications/

Also here it works as expected:

Termination: CONVERGENCE (Function tolerance reached. |cost_change|/cost: 8.272755e-05 <= 1.000000e-04)

Mean reprojection error 1.00979 number residuals: 227334 I0108 18:02:28.883790 269841 continuous_time_imu_to_camera_calibration.cc:222] Mean reprojection error 1.00979px I0108 18:02:28.883800 269841 continuous_time_imu_to_camera_calibration.cc:223] Mean reprojection error after line delay optim 1.00979px g: 0.422354 -9.82961 0.127258 accel_bias at time 0: -0.0288142 -0.0365272 0.0100668 gyro_bias at time 0: -0.00160678 0.00187235 -0.00758646 T_i_c qw,qx,qy,qz: 0.00023031 -0.00261572 -0.709489 0.704712 T_i_c t: 0.00322658 -0.0197022 0.00300103 T_i_c R: -0.999986 0.00338705 -0.00401347 0.00403626 0.00674939 -0.999969 -0.00335986 -0.999971 -0.00676297 Initialized line delay [us]: 30.8951 Calibrated line delay [us]: 30.8951

Thanks a lot! I tried to build the newest update and now it can converge after 5 iterations :)

I have another question related to the ORB_SLAM fork. I attempted to execute VI-ORB_SLAM3 using the GoPro9 dataset, but it consistently fails to initialize the IMU. I tried to update the IMU parameters from OpenICC but still not able to initialize IMU. Could this issue be related to the json file generated from telemetry extractor script? here is the log from my setting: log_ORB_SLAM3.txt

[fickrie67]

I just also verified with my local build that it works: (base) steffen@steffen-IdeaPad-5-Pro-16ARH7:~/projects/OpenImuCameraCalibrator$ python python/run_gopro_calibration.py --path_calib_dataset /home/steffen/Downloads/GoPro9/dataset3/ --path_to_build ./build/applications/

Also here it works as expected:

Termination: CONVERGENCE (Function tolerance reached. |cost_change|/cost: 8.272755e-05 <= 1.000000e-04)

Mean reprojection error 1.00979 number residuals: 227334 I0108 18:02:28.883790 269841 continuous_time_imu_to_camera_calibration.cc:222] Mean reprojection error 1.00979px I0108 18:02:28.883800 269841 continuous_time_imu_to_camera_calibration.cc:223] Mean reprojection error after line delay optim 1.00979px g: 0.422354 -9.82961 0.127258 accel_bias at time 0: -0.0288142 -0.0365272 0.0100668 gyro_bias at time 0: -0.00160678 0.00187235 -0.00758646 T_i_c qw,qx,qy,qz: 0.00023031 -0.00261572 -0.709489 0.704712 T_i_c t: 0.00322658 -0.0197022 0.00300103 T_i_c R: -0.999986 0.00338705 -0.00401347 0.00403626 0.00674939 -0.999969 -0.00335986 -0.999971 -0.00676297 Initialized line delay [us]: 30.8951 Calibrated line delay [us]: 30.8951

Thanks a lot! I tried to build the newest update and now it can converge after 5 iterations :)

I have another question related to the ORB_SLAM fork. I attempted to execute VI-ORB_SLAM3 using the GoPro9 dataset, but it consistently fails to initialize the IMU. I tried to update the IMU parameters from OpenICC but still not able to initialize IMU. Could this issue be related to the json file generated from telemetry extractor script?

here is the log from my setting:

log_ORB_SLAM3.txt

I discovered that I needed to use the correct camera intrinsic parameters from the calibration result, and then it worked :)

[urbste]

Closing as solved :)