search

urbste / OpenImuCameraCalibrator

Camera calibration tool
GNU Affero General Public License v3.0
197 stars 43 forks source link

E0610 11:26:11.893433 23973 trust_region_minimizer.cc:73] Terminating: Residual and Jacobian evaluation failed. Calibration failed. #4

Closed laldev closed 2 years ago

laldev commented 3 years ago

Hi, Any help on the following error. Thanks in advance.

pc@pc:~/work/vendor/linux/OpenImuCameraCalibrator$ python3 python/run_gopro_calibration.py --path_calib_dataset=/home/EU/USER/work/vendor/data/gopro/GoPro9/dataset3/ --checker_size_m=0.021 --image_downsample_factor=2 --camera_model=PINHOLE  --path_to_build=/home/EU/USER/work/vendor/linux/OpenImuCameraCalibrator/build/applications/ --path_to_src=/home/EU/USER/work/vendor/linux/OpenImuCameraCalibrator/
['/home/EU/USER/work/vendor/data/gopro/GoPro9/dataset3/cam/GH010162.MP4']
==================================================================
Running corner extraction.
==================================================================
Extracing corners for camera calibration.
I0610 11:22:34.732122 23971 extract_board_to_json.cc:57] Skipping corner extraction. Already extracted for: /home/EU/USER/work/vendor/data/gopro/GoPro9/dataset3/cam/GH010162.MP4
Extracing corners for imu camera calibration.
I0610 11:22:34.778587 23972 extract_board_to_json.cc:57] Skipping corner extraction. Already extracted for: /home/EU/USER/work/vendor/data/gopro/GoPro9/dataset3/cam_imu/GH010164.MP4
Finished corner extraction.
==================================================================
Corner extraction took 0.09s.
==================================================================
==================================================================
Running camera calibration.
==================================================================
Calibrating camera.
View: 0/1463 initialized for calibration.
View: 100/1463 initialized for calibration.
View: 200/1463 initialized for calibration.
View: 300/1463 initialized for calibration.
View: 400/1463 initialized for calibration.
View: 500/1463 initialized for calibration.
View: 600/1463 initialized for calibration.
View: 700/1463 initialized for calibration.
View: 800/1463 initialized for calibration.
View: 900/1463 initialized for calibration.
View: 1000/1463 initialized for calibration.
View: 1100/1463 initialized for calibration.
View: 1200/1463 initialized for calibration.
View: 1300/1463 initialized for calibration.
View: 1400/1463 initialized for calibration.
E0610 11:26:11.893433 23973 trust_region_minimizer.cc:73] Terminating: Residual and Jacobian evaluation failed.
E0610 11:26:11.898849 23973 trust_region_minimizer.cc:73] Terminating: Residual and Jacobian evaluation failed.
Not enough views left for proper calibration!
E0610 11:26:11.898969 23973 camera_calibrator.cc:307] Calibration failed.
Focal Length:482.086px Principal Point: 480/270px.
Finished camera calibration.
==================================================================
Camera calibration took 217.15s.
==================================================================
==================================================================
Extracting GoPro telemetry for imu bias and camera imu calibration.
==================================================================
Input video path: /home/EU/USER/work/vendor/data/gopro/GoPro9/dataset3/cam_imu/GH010164.MP4
Output file will be: /home/EU/USER/work/vendor/data/gopro/GoPro9/dataset3/cam_imu/GH010164.MP4_telemetry.json
Skipping GH010164 as telemetry file already exists: /home/EU/USER/work/vendor/data/gopro/GoPro9/dataset3/cam_imu/GH010164.json
Done
==================================================================
Telemetry extraction took 0.13s.
==================================================================
Input video path: /home/EU/USER/work/vendor/data/gopro/GoPro9/dataset3/imu_bias/GH010163.MP4
Output file will be: /home/EU/USER/work/vendor/data/gopro/GoPro9/dataset3/imu_bias/GH010163.MP4_telemetry.json
Skipping GH010163 as telemetry file already exists: /home/EU/USER/work/vendor/data/gopro/GoPro9/dataset3/imu_bias/GH010163.json
Done
==================================================================
Estimating IMU biases.
==================================================================
Estimated biases:
gyroscope bias:     -0.00161 rad/s, 0.00187 rad/s, -0.00759 rad/s
accelerometer bias: -0.02881 m/s2,  -0.03653 m/s2,  0.01007 m/s2
Writing result to:  /home/EU/USER/work/vendor/data/gopro/GoPro9/dataset3/imu_bias/imu_bias_GH010163.json
==================================================================
IMU bias estimation took 0.41s.
==================================================================
==================================================================
Estimating camera poses for IMU - CAM calibration.
==================================================================
Could not open: /home/EU/USER/work/vendor/data/gopro/GoPro9/dataset3/cam/cam_calib_GH010162_ph_2.0.json
F0610 11:26:13.795512 24156 estimate_camera_poses_from_checkerboard.cc:52] Check failed: read_camera_calibration(FLAGS_camera_calibration_json, camera, fps) Could not read camera calibration: /home/EU/USER/work/vendor/data/gopro/GoPro9/dataset3/cam/cam_calib_GH010162_ph_2.0.json
*** Check failure stack trace: ***
    @     0x7fa7420b90cd  google::LogMessage::Fail()
    @     0x7fa7420baf33  google::LogMessage::SendToLog()
    @     0x7fa7420b8c28  google::LogMessage::Flush()
    @     0x7fa7420bb999  google::LogMessageFatal::~LogMessageFatal()
    @     0x55619acb5dab  (unknown)
    @     0x7fa73fed1bf7  __libc_start_main
    @     0x55619acb56aa  (unknown)
==================================================================
Pose estimation estimation took 0.98s.
==================================================================
==================================================================
Estimating Spline error weighting and knot spacing.
==================================================================
Knot spacing SO3:               0.050 seconds at quality level q_so3=0.99
Knot spacing  R3:               0.090 seconds at quality level q_r3=0.99
Gyroscope weighting factor:     22.374 at quality level q_so3=0.99
Accelerometer weighting factor: 3.852 at quality level q_r3=0.99
Writing result to:  /home/EU/USER/work/vendor/data/gopro/GoPro9/dataset3/cam_imu/spline_info_GH010164.json
==================================================================
Spline weighting and knot spacing estimation took 0.83s.
==================================================================
==================================================================
Initializing IMU to camera rotation.
==================================================================
E0610 11:26:14.644371 24181 reconstruction_reader.cc:65] Could not open the file: /home/EU/USER/work/vendor/data/gopro/GoPro9/dataset3/cam_imu/pose_calib_GH010164.calibdata for reading.
F0610 11:26:14.644461 24181 estimate_imu_to_camera_rotation.cc:55] Check failed: theia::ReadReconstruction(FLAGS_input_pose_calibration_dataset, &pose_dataset) 
*** Check failure stack trace: ***
    @     0x7f58705300cd  google::LogMessage::Fail()
    @     0x7f5870531f33  google::LogMessage::SendToLog()
    @     0x7f587052fc28  google::LogMessage::Flush()
    @     0x7f5870532999  google::LogMessageFatal::~LogMessageFatal()
    @     0x55d6c36d0bb0  (unknown)
    @     0x7f586ed6cbf7  __libc_start_main
    @     0x55d6c36d052a  (unknown)
==================================================================
Spline weighting and knot spacing estimation took 0.01s.
==================================================================
==================================================================
Optimizing IMU to Camera calibration using Spline Fusion.
==================================================================
WARNING: Logging before InitGoogleLogging() is written to STDERR
E0610 11:26:14.685009 24182 reconstruction_reader.cc:65] Could not open the file: /home/EU/USER/work/vendor/data/gopro/GoPro9/dataset3/cam_imu/pose_calib_GH010164.calibdata for reading.
F0610 11:26:14.685065 24182 continuous_time_imu_to_camera_calibration.cc:87] Check failed: theia::ReadReconstruction(FLAGS_input_pose_dataset, &pose_dataset) Could not read Reconstruction file.
*** Check failure stack trace: ***
==================================================================
Spline weighting and knot spacing estimation took 0.04s.
==================================================================
==================================================================
Print results.
==================================================================
Traceback (most recent call last):
  File "/home/EU/USER/work/vendor/linux/OpenImuCameraCalibrator/python/print_result_stats.py", line 75, in <module>
    main()
  File "/home/EU/USER/work/vendor/linux/OpenImuCameraCalibrator/python/print_result_stats.py", line 24, in main
    data = read_calib_json(args.path_results)
  File "/home/EU/USER/work/vendor/linux/OpenImuCameraCalibrator/python/print_result_stats.py", line 9, in read_calib_json
    with open(file, 'r') as f:
FileNotFoundError: [Errno 2] No such file or directory: '/home/EU/USER/work/vendor/data/gopro/GoPro9/dataset3/cam_imu/cam_imu_calib_result_GH010164.json'
pc@pc:~/work/vendor/linux/OpenImuCameraCalibrator$ ls /home/EU/USER/work/vendor/data/gopro/GoPro9/dataset3/cam/
cam_calib_GH010162_ph_2.0_ransac_pose.ply  cam_corners_GH010162.uson  GH010162.MP4
urbste commented 3 years ago

Hi, I think you are using the wrong camera model. Pinhole can only be used, iff you know that your camera has literally no distortion. (e.g. for a smartphone or a GoPro in Linear mode). However, as the example dataset is recorded with strong distortion, you will need to use a different distortion model (Double Sphere, Fisheye, Division Undistortion, Extended Unified).

Btw: thanks for taking all the pitfalls during installation and sticking with it :-D. I will try to update the readme and the build process to make it easier.

urbste commented 2 years ago

Any news on this? Was it the camera model? I would close the issue for now. Feel free to reopen it.