urbste / OpenImuCameraCalibrator

Camera calibration tool
GNU Affero General Public License v3.0
212 stars 47 forks source link

JSON files not being generated for GoPro10 #49

Open Peungwon opened 1 month ago

Peungwon commented 1 month ago

IMU bias seems to work but I'm getting a JSON related issue for cam and cam_imu. Error message below. I've uploaded my calibration data here: https://www.dropbox.com/scl/fi/uuirwlxh3w3srhf1p2uzq/GoPro10.zip?rlkey=gqz3jq9zme25r4rhvnscrttng&st=gd5txrpg&dl=0

Thank you

tj@tj-MG-VCP2-15I3080T:~/OpenImuCameraCalibrator/python$ python run_gopro_calibration.py --path_calib_dataset=/home/tj/datasets/GoPro10/dataset --checker_size_m=0.05715 --image_downsample_factor=2 --camera_model=FISHEYE ['/home/tj/datasets/GoPro10/dataset/cam/GX010017.MP4']

Running corner extraction.

Extracing corners for camera calibration. I0726 14:29:54.319873 331521 extract_board_to_json.cc:92] Starting board extraction. This might take a while... Total number of frames: 820 I0726 14:29:55.068872 331521 board_extractor.cc:443] Extracting corners from frame 60 / 820 I0726 14:29:55.769227 331521 board_extractor.cc:443] Extracting corners from frame 120 / 820 I0726 14:29:56.485342 331521 board_extractor.cc:443] Extracting corners from frame 180 / 820 I0726 14:29:57.186159 331521 board_extractor.cc:443] Extracting corners from frame 240 / 820 I0726 14:29:57.883438 331521 board_extractor.cc:443] Extracting corners from frame 300 / 820 I0726 14:29:58.597607 331521 board_extractor.cc:443] Extracting corners from frame 360 / 820 I0726 14:29:59.336733 331521 board_extractor.cc:443] Extracting corners from frame 420 / 820 I0726 14:30:00.067791 331521 board_extractor.cc:443] Extracting corners from frame 480 / 820 I0726 14:30:00.787941 331521 board_extractor.cc:443] Extracting corners from frame 540 / 820 I0726 14:30:01.538920 331521 board_extractor.cc:443] Extracting corners from frame 600 / 820 I0726 14:30:02.250234 331521 board_extractor.cc:443] Extracting corners from frame 660 / 820 I0726 14:30:03.009642 331521 board_extractor.cc:443] Extracting corners from frame 720 / 820 I0726 14:30:03.734330 331521 board_extractor.cc:443] Extracting corners from frame 780 / 820 Extracing corners for imu camera calibration. I0726 14:30:04.338131 331612 extract_board_to_json.cc:92] Starting board extraction. This might take a while... Total number of frames: 1405 I0726 14:30:05.111332 331612 board_extractor.cc:443] Extracting corners from frame 60 / 1405 I0726 14:30:05.851434 331612 board_extractor.cc:443] Extracting corners from frame 120 / 1405 I0726 14:30:06.587872 331612 board_extractor.cc:443] Extracting corners from frame 180 / 1405 I0726 14:30:07.327621 331612 board_extractor.cc:443] Extracting corners from frame 240 / 1405 I0726 14:30:08.058885 331612 board_extractor.cc:443] Extracting corners from frame 300 / 1405 I0726 14:30:08.800006 331612 board_extractor.cc:443] Extracting corners from frame 360 / 1405 I0726 14:30:09.539168 331612 board_extractor.cc:443] Extracting corners from frame 420 / 1405 I0726 14:30:10.282332 331612 board_extractor.cc:443] Extracting corners from frame 480 / 1405 I0726 14:30:11.018980 331612 board_extractor.cc:443] Extracting corners from frame 540 / 1405 I0726 14:30:11.756665 331612 board_extractor.cc:443] Extracting corners from frame 600 / 1405 I0726 14:30:12.502490 331612 board_extractor.cc:443] Extracting corners from frame 660 / 1405 I0726 14:30:13.239267 331612 board_extractor.cc:443] Extracting corners from frame 720 / 1405 I0726 14:30:13.970516 331612 board_extractor.cc:443] Extracting corners from frame 780 / 1405 I0726 14:30:14.686739 331612 board_extractor.cc:443] Extracting corners from frame 840 / 1405 I0726 14:30:15.439325 331612 board_extractor.cc:443] Extracting corners from frame 900 / 1405 I0726 14:30:16.214404 331612 board_extractor.cc:443] Extracting corners from frame 960 / 1405 I0726 14:30:16.946134 331612 board_extractor.cc:443] Extracting corners from frame 1020 / 1405 I0726 14:30:17.776433 331612 board_extractor.cc:443] Extracting corners from frame 1080 / 1405 I0726 14:30:18.611075 331612 board_extractor.cc:443] Extracting corners from frame 1140 / 1405 I0726 14:30:19.409662 331612 board_extractor.cc:443] Extracting corners from frame 1200 / 1405 I0726 14:30:20.165485 331612 board_extractor.cc:443] Extracting corners from frame 1260 / 1405 I0726 14:30:20.884739 331612 board_extractor.cc:443] Extracting corners from frame 1320 / 1405 I0726 14:30:21.626439 331612 board_extractor.cc:443] Extracting corners from frame 1380 / 1405 Finished corner extraction.

Corner extraction took 27.73s.

================================================================== Running camera calibration.

Calibrating camera. calibrate_camera: /home/tj/OpenImuCameraCalibrator/include/OpenCameraCalibrator/utils/json.h:17346: const value_type& nlohmann::basic_json<ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberUnsignedType, NumberFloatType, AllocatorType, JSONSerializer>::operator const [with T = const char; ObjectType = std::map; ArrayType = std::vector; StringType = std::__cxx11::basic_string; BooleanType = bool; NumberIntegerType = long int; NumberUnsignedType = long unsigned int; NumberFloatType = double; AllocatorType = std::allocator; JSONSerializer = nlohmann::adl_serializer; nlohmann::basic_json<ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberUnsignedType, NumberFloatType, AllocatorType, JSONSerializer>::const_reference = const nlohmann::basic_json<>&; nlohmann::basic_json<ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberUnsignedType, NumberFloatType, AllocatorType, JSONSerializer>::value_type = nlohmann::basic_json<>]: Assertion `m_value.object->find(key) != m_value.object->end()' failed. Finished camera calibration.

Camera calibration took 0.13s.

================================================================== Extracting GoPro telemetry for imu bias and camera imu calibration.

================================================================== Telemetry extraction took 0.11s.

================================================================== Estimating IMU biases.

Estimated biases: gyroscope bias: 0.00183 rad/s, -0.00239 rad/s, -0.00144 rad/s accelerometer bias: -0.17995 m/s2, 0.28617 m/s2, 0.00697 m/s2 Writing result to: /home/tj/datasets/GoPro10/dataset/imu_bias/imu_bias_GX010002.json

IMU bias estimation took 0.42s.

================================================================== Estimating camera poses for IMU - CAM calibration.

Could not open: /home/tj/datasets/GoPro10/dataset/cam/cam_calib_GX010017_fi_2.0.json F0726 14:30:22.770514 331791 estimate_camera_poses_from_checkerboard.cc:54] Check failed: read_camera_calibration(FLAGS_camera_calibration_json, camera, fps) Could not read camera calibration: /home/tj/datasets/GoPro10/dataset/cam/cam_calib_GX010017_fi_2.0.json Check failure stack trace: @ 0x7ff86347b1c3 google::LogMessage::Fail() @ 0x7ff86348025b google::LogMessage::SendToLog() @ 0x7ff86347aebf google::LogMessage::Flush() @ 0x7ff86347b6ef google::LogMessageFatal::~LogMessageFatal() @ 0x55ff301a2648 main @ 0x7ff82d477083 __libc_start_main @ 0x55ff301a200e _start

Pose estimation estimation took 0.12s.

================================================================== Estimating Spline error weighting and knot spacing.

Knot spacing SO3: 0.042 seconds at quality level q_so3=0.99 Knot spacing R3: 0.037 seconds at quality level q_r3=0.99 Gyroscope weighting factor: 31.182 at quality level q_so3=0.99 Accelerometer weighting factor: 4.602 at quality level q_r3=0.99 Writing result to: /home/tj/datasets/GoPro10/dataset/cam_imu/spline_info_GX010018.json

Spline weighting and knot spacing estimation took 0.63s.

================================================================== Initializing IMU to camera rotation.

E0726 14:30:23.516996 331862 reconstruction_reader.cc:65] Could not open the file: /home/tj/datasets/GoPro10/dataset/cam_imu/pose_calib_GX010018.calibdata for reading. F0726 14:30:23.517024 331862 estimate_imu_to_camera_rotation.cc:62] Check failed: theia::ReadReconstruction(FLAGS_input_pose_calibration_dataset, &pose_dataset) Check failure stack trace: @ 0x7ff552b2d1c3 google::LogMessage::Fail() @ 0x7ff552b3225b google::LogMessage::SendToLog() @ 0x7ff552b2cebf google::LogMessage::Flush() @ 0x7ff552b2d6ef google::LogMessageFatal::~LogMessageFatal() @ 0x55a87b38ab52 main @ 0x7ff551578083 __libc_start_main @ 0x55a87b38a5fe _start

Spline weighting and knot spacing estimation took 0.11s.

================================================================== Optimizing IMU to Camera calibration using Spline Fusion.

WARNING: Logging before InitGoogleLogging() is written to STDERR E0726 14:30:23.642426 331864 reconstruction_reader.cc:65] Could not open the file: /home/tj/datasets/GoPro10/dataset/cam_imu/pose_calib_GX010018.calibdata for reading. F0726 14:30:23.642457 331864 continuous_time_imu_to_camera_calibration.cc:96] Check failed: theia::ReadReconstruction(FLAGS_input_pose_dataset, &pose_dataset) Could not read Reconstruction file. Check failure stack trace:

Spline weighting and knot spacing estimation took 0.12s.

================================================================== Print results.

Traceback (most recent call last): File "/home/tj/OpenImuCameraCalibrator/python/../python/print_result_stats.py", line 98, in main() File "/home/tj/OpenImuCameraCalibrator/python/../python/print_result_stats.py", line 24, in main data = read_calib_json(args.path_results) File "/home/tj/OpenImuCameraCalibrator/python/../python/print_result_stats.py", line 9, in read_calib_json with open(file, 'r') as f: FileNotFoundError: [Errno 2] No such file or directory: '/home/tj/datasets/GoPro10/dataset/cam_imu/cam_imu_calib_result_GX010018.json'

Peungwon commented 1 month ago

I notice that my GoPro10 issue is identical to this self-closed error report: https://github.com/urbste/OpenImuCameraCalibrator/issues/40

Does this project support GoPro10?