search

rameau-fr / MC-Calib

A generic and robust calibration toolbox for multi-camera systems
MIT License
377 stars 54 forks source link

Problems with calibrating intrinsic parameters on smartphone camera. Update: issue was wrong charuco board config, 4x3 instead of 3x4 #50

Closed DawidStankiewicz closed 8 months ago

DawidStankiewicz commented 8 months ago

System information (version)

Vision system

######################################## Boards Parameters ################################################### number_x_square: 4 # number of squares in the X direction number_y_square: 3 # number of squares the Y direction resolution_x: 3840 # horizontal resolution in pixel resolution_y: 2160 # vertical resolution in pixel length_square: 0.04 # parameters on the marker (can be kept as it is) length_marker: 0.03 # parameters on the marker (can be kept as it is) number_board: 1 # number of boards used for calibration (for overlapping camera 1 is enough ...) boards_index: [ ] # leave it empty [] if the board index are ranging from zero to number_board; example of usage boards_index: [5,10 <-- only two board with index 5/10 square_size: 6.5 # size of each square of the board in cm/mm/whatever you want

############# Boards Parameters for different board size (leave empty if all boards have the same size) ################# number_x_square_per_board: [ ] number_y_square_per_board: [ ] square_size_per_board: [ ]

######################################## Camera Parameters ################################################### distortion_model: 0 # 0:Brown (perspective) // 1: Kannala (fisheye) distortion_per_camera: [ ] # specify the model per camera, #leave "distortion_per_camera" empty [] if they all follow the same model (make sure that the vector is as long as cameras nb) number_camera: 1 # number of cameras in the rig to calibrate refine_corner: 1 # activate or deactivate the corner refinement min_perc_pts: 0.8 # min percentage of points visible to assume a good detection

cam_params_path: None # file with cameras intrinsics to initialize the intrinsic, write "None" if no initialization available fix_intrinsic: 0 #if 1 then the intrinsic parameters will not be estimated nor refined (initial value needed) ######################################## Images Parameters ################################################### camprefix: "Cam"

######################################## Optimization Parameters ############################################# ransac_threshold: 10 # RANSAC threshold in pixel (keep it high just to remove strong outliers) number_iterations: 1000 # Max number of iterations for the non linear refinement

######################################## Hand-eye method ############################################# he_approach: 0 #0: bootstrapped he technique, 1: traditional he

######################################## Output Parameters ################################################### save_detection: 1 save_reprojection: 1 camera_params_file_name: "cameras.yml" # "name.yml"

* Image sequences => pass `QSp34wGmuD`
 [https://nextcloud.sportvision.tech/s/KSxPWpji9BF8ytj](https://nextcloud.sportvision.tech/s/KSxPWpji9BF8ytj) - there are source images and example output.

### Describe the issue / bug

I'm trying to calibrate intrinsic parameters of the smartphone camera using MC-Calib. App is often crashing and when it's not then results and logs from calibration are weird. I'm using board 4x3, so it has 6 points. Log says that detector found 6 points in all images (134):

0000285 | 2023-11-09, 14:37:35.491338 [info] - Number of detected corners :: 6 0000286 | 2023-11-09, 14:37:35.526970 [info] - Number of detected corners :: 6 0000287 | 2023-11-09, 14:37:35.532342 [info] - Number of detected corners :: 6 0000288 | 2023-11-09, 14:37:35.725408 [info] - Board extraction done! 0000289 | 2023-11-09, 14:37:35.725435 [info] - Intrinsic calibration initiated 0000290 | 2023-11-09, 14:37:35.725441 [info] - Initializing camera calibration using images 0000291 | 2023-11-09, 14:37:35.725449 [info] - NB of board available in this camera :: 134 0000292 | 2023-11-09, 14:37:35.725457 [info] - NB of frames where this camera saw a board :: 134 0000293 | 2023-11-09, 14:37:52.858604 [info] - cameraMatrix : [730.5722449957763, 0, 1842.225576510645; 0, 103.8567137340529, 1010.620326430428; 0, 0, 1] 0000294 | 2023-11-09, 14:37:52.858662 [info] - distCoeffs : [-0.000145485069557112, -9.341887444497364e-07, -0.001914292531187723, -0.00490921035096062, -8.294359195743176e-08] 0000295 | 2023-11-09, 14:37:52.860456 [debug] - Trans :: [3.758818329998107; 7.510510097752508; 8.679588315941288] Rot :: [-1.192318438958881; 1.614630443992688; -0.9228510025860922]


But then I see some calculations are done on different number of points than 6:

0005110 | 2023-11-09, 14:38:20.715560 [debug] - cam_id :: 0 object_id :: 0 mean error :: 29.6129 nb pts :: 3 0005111 | 2023-11-09, 14:38:20.715595 [debug] - cam_id :: 0 object_id :: 0 mean error :: 27.1408 nb pts :: 3 0005112 | 2023-11-09, 14:38:20.715629 [debug] - cam_id :: 0 object_id :: 0 mean error :: 81.2194 nb pts :: 3 0005113 | 2023-11-09, 14:38:20.715668 [debug] - cam_id :: 0 object_id :: 0 mean error :: 66.0296 nb pts :: 3 0005114 | 2023-11-09, 14:38:20.715703 [debug] - cam_id :: 0 object_id :: 0 mean error :: 88.8366 nb pts :: 3 0005115 | 2023-11-09, 14:38:20.715740 [debug] - cam_id :: 0 object_id :: 0 mean error :: 291.77 nb pts :: 3 0005116 | 2023-11-09, 14:38:20.715775 [debug] - cam_id :: 0 object_id :: 0 mean error :: 229.415 nb pts :: 3 0005117 | 2023-11-09, 14:38:20.715811 [debug] - cam_id :: 0 object_id :: 0 mean error :: 217.277 nb pts :: 3 0005118 | 2023-11-09, 14:38:20.715848 [debug] - cam_id :: 0 object_id :: 0 mean error :: 215.845 nb pts :: 3 0005119 | 2023-11-09, 14:38:20.715882 [debug] - cam_id :: 0 object_id :: 0 mean error :: 581.92 nb pts :: 4 0005120 | 2023-11-09, 14:38:20.715920 [debug] - cam_id :: 0 object_id :: 0 mean error :: 590.8 nb pts :: 4 0005121 | 2023-11-09, 14:38:20.715955 [debug] - cam_id :: 0 object_id :: 0 mean error :: 287.132 nb pts :: 3 0005122 | 2023-11-09, 14:38:20.715967 [info] - Final refinement done 0005123 | 2023-11-09, 14:38:38.345637 [info] - Save parameters 0005124 | 2023-11-09, 14:38:38.356766 [info] - mean reprojection error :: 175.409 0005125 | 2023-11-09, 14:38:38.360028 [info] - Calibration took 72 seconds


I'm using a subset size of 150 frames to get these results.

Sometimes the script produces some weird output like above, but more often it's failing due to this error:

0001480 | 2023-11-09, 14:56:08.856804 [info] - Frame :: 169 Boards :: 0 --- Mean Error ::12.0435 Nb pts :: 2 0001481 | 2023-11-09, 14:56:08.856821 [info] - High error detected in this board pose estimation 0001482 | 2023-11-09, 14:56:08.856827 [info] - Frame :: 161 Boards :: 0 --- Mean Error ::10.9782 Nb pts :: 2 0001483 | 2023-11-09, 14:56:08.856844 [info] - High error detected in this board pose estimation 0001484 | 2023-11-09, 14:56:08.856849 [info] - Frame :: 167 Boards :: 0 --- Mean Error ::12.6696 Nb pts :: 2 0001485 | 2023-11-09, 14:56:08.856867 [info] - High error detected in this board pose estimation 0001486 | 2023-11-09, 14:56:08.856872 [info] - Frame :: 168 Boards :: 0 --- Mean Error ::11.8517 Nb pts :: 2 0001487 | 2023-11-09, 14:56:08.856898 [info] - High error detected in this board pose estimation 0001488 | 2023-11-09, 14:56:08.856904 [info] - Frame :: 171 Boards :: 0 --- Mean Error ::10.6186 Nb pts :: 2 terminate called after throwing an instance of 'cv::Exception' what(): OpenCV(4.2.0) ../modules/core/src/matrix_expressions.cpp:23: error: (-5:Bad argument) Matrix operand is an empty matrix. in function 'checkOperandsExist'

bash: line 1: 51 Aborted (core dumped) ./build/apps/calibrate/calibrate /home/sportv03/MC-Calib/data/subsets/subset_01/calib_param.yml


I don't know why. What am I doing wrong?

Does 4x3 charuco board is valid for this calibration? I know that 4 points are minimum for calculations, but maybe this number is higher for some reason? 

Extrinsic calibration works on synthetic data on this 4x3 charuco board (intrinsic calibration is defined by me). 

It would be great to have more configuration validation or data validation before each step of the script, so it would be more obvious to everyone why something is not working.
DawidStankiewicz commented 8 months ago

This image frameau/opencv-ceres seems old, I don't know why it was in the code, but when I switched to this bailool/mc-calib-prod, then I get same error I sometimes got before:

0000927 | 2023-11-09, 19:24:45.767402 [info] - High error detected in this board pose estimation
0000928 | 2023-11-09, 19:24:45.767409 [info] - Frame :: 141  Boards :: 0  --- Mean Error ::141.812  Nb pts :: 3
0000929 | 2023-11-09, 19:24:45.767429 [info] - High error detected in this board pose estimation
0000930 | 2023-11-09, 19:24:45.767435 [info] - Frame :: 146  Boards :: 0  --- Mean Error ::303.462  Nb pts :: 3
0000931 | 2023-11-09, 19:24:45.767473 [info] - High error detected in this board pose estimation
0000932 | 2023-11-09, 19:24:45.767480 [info] - Frame :: 147  Boards :: 0  --- Mean Error ::311.97  Nb pts :: 3
0000933 | 2023-11-09, 19:24:45.767500 [info] - High error detected in this board pose estimation
0000934 | 2023-11-09, 19:24:45.767506 [info] - Frame :: 149  Boards :: 0  --- Mean Error ::335.033  Nb pts :: 3
0000935 | 2023-11-09, 19:24:45.767529 [info] - High error detected in this board pose estimation
0000936 | 2023-11-09, 19:24:45.767535 [info] - Frame :: 148  Boards :: 0  --- Mean Error ::313.325  Nb pts :: 3
0000937 | 2023-11-09, 19:24:45.767543 [info] - Intrinsic Calibration done!
0000938 | 2023-11-09, 19:24:45.767549 [info] - 3D Object calibration initiated
=================================================================
==19==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x602000901b1c at pc 0x555e2e3ea891 bp 0x7ffeac988490 sp 0x7ffeac988480
READ of size 4 at 0x602000901b1c thread T0
    #0 0x555e2e3ea890 in ransacP3P(std::vector<cv::Point3_<float>, std::allocator<cv::Point3_<float> > > const&, std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > > const&, cv::Mat, cv::Mat, cv::Mat&, cv::Mat&, float, int, double, bool) /home/sportv03/MC-Calib/McCalib/src/geometrytools.cpp:252
    #1 0x555e2e3eceb8 in ransacP3PDistortion(std::vector<cv::Point3_<float>, std::allocator<cv::Point3_<float> > > const&, std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > > const&, cv::Mat, cv::Mat, cv::Mat&, cv::Mat&, float, int, int, double, bool) /home/sportv03/MC-Calib/McCalib/src/geometrytools.cpp:576
    #2 0x555e2e3d4c5e in Object3DObs::estimatePose(float, int) /home/sportv03/MC-Calib/McCalib/src/Object3DObs.cpp:237
    #3 0x555e2e289ee2 in McCalib::Calibration::estimatePoseAllObjects() /home/sportv03/MC-Calib/McCalib/src/McCalib.cpp:979
    #4 0x555e2e2b5fc5 in McCalib::Calibration::calibrate3DObjects() /home/sportv03/MC-Calib/McCalib/src/McCalib.cpp:2084
    #5 0x555e2e2301c5 in runCalibrationWorkflow(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) /home/sportv03/MC-Calib/apps/calibrate/src/calibrate.cpp:21
    #6 0x555e2e225c15 in main /home/sportv03/MC-Calib/apps/calibrate/src/calibrate.cpp:97
    #7 0x7f9ad5c95082 in __libc_start_main (/usr/lib/x86_64-linux-gnu/libc.so.6+0x24082)
    #8 0x555e2e22dcdd in _start (/home/sportv03/MC-Calib/build/apps/calibrate/calibrate+0xb7cdd)

0x602000901b1c is located 0 bytes to the right of 12-byte region [0x602000901b10,0x602000901b1c)
allocated by thread T0 here:
    #0 0x7f9ad8f4f587 in operator new(unsigned long) ../../../../src/libsanitizer/asan/asan_new_delete.cc:104
    #1 0x555e2e3e5949 in __gnu_cxx::new_allocator<int>::allocate(unsigned long, void const*) /usr/include/c++/9/ext/new_allocator.h:114
    #2 0x555e2e3e5949 in std::allocator_traits<std::allocator<int> >::allocate(std::allocator<int>&, unsigned long) /usr/include/c++/9/bits/alloc_traits.h:443
    #3 0x555e2e3e5949 in std::_Vector_base<int, std::allocator<int> >::_M_allocate(unsigned long) /usr/include/c++/9/bits/stl_vector.h:343
    #4 0x555e2e3e5949 in std::_Vector_base<int, std::allocator<int> >::_M_create_storage(unsigned long) /usr/include/c++/9/bits/stl_vector.h:358
    #5 0x555e2e3e5949 in std::_Vector_base<int, std::allocator<int> >::_Vector_base(unsigned long, std::allocator<int> const&) /usr/include/c++/9/bits/stl_vector.h:302
    #6 0x555e2e3e5949 in std::vector<int, std::allocator<int> >::vector(unsigned long, std::allocator<int> const&) /usr/include/c++/9/bits/stl_vector.h:508
    #7 0x555e2e3e5949 in ransacP3P(std::vector<cv::Point3_<float>, std::allocator<cv::Point3_<float> > > const&, std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > > const&, cv::Mat, cv::Mat, cv::Mat&, cv::Mat&, float, int, double, bool) /home/sportv03/MC-Calib/McCalib/src/geometrytools.cpp:243
    #8 0x555e2e3eceb8 in ransacP3PDistortion(std::vector<cv::Point3_<float>, std::allocator<cv::Point3_<float> > > const&, std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > > const&, cv::Mat, cv::Mat, cv::Mat&, cv::Mat&, float, int, int, double, bool) /home/sportv03/MC-Calib/McCalib/src/geometrytools.cpp:576
    #9 0x555e2e3d4c5e in Object3DObs::estimatePose(float, int) /home/sportv03/MC-Calib/McCalib/src/Object3DObs.cpp:237
    #10 0x555e2e289ee2 in McCalib::Calibration::estimatePoseAllObjects() /home/sportv03/MC-Calib/McCalib/src/McCalib.cpp:979

SUMMARY: AddressSanitizer: heap-buffer-overflow /home/sportv03/MC-Calib/McCalib/src/geometrytools.cpp:252 in ransacP3P(std::vector<cv::Point3_<float>, std::allocator<cv::Point3_<float> > > const&, std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > > const&, cv::Mat, cv::Mat, cv::Mat&, cv::Mat&, float, int, double, bool)
Shadow bytes around the buggy address:
  0x0c0480118310: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c0480118320: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c0480118330: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c0480118340: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c0480118350: fa fa fa fa fa fa 00 04 fa fa 04 fa fa fa 04 fa
=>0x0c0480118360: fa fa 00[04]fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c0480118370: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c0480118380: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c0480118390: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c04801183a0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c04801183b0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
  Shadow gap:              cc
==19==ABORTING

If it's geometrytools.cpp:252 then it looks like a problem with not enough points 🤔 but still log says that on all images detected 6 points.

    // pick 4 points
    std::random_shuffle(myvector.begin(), myvector.end());
    std::array<int, 4> idx = {myvector[0], myvector[1], myvector[2],
                              myvector[3]};
DawidStankiewicz commented 8 months ago

Ok it was simpler than it looked. Here is the problem:

######################################## Boards Parameters ###################################################
number_x_square: 4         # number of squares in the X direction
number_y_square: 3         # number of squares the Y direction

after switching to 3x4 it's working

######################################## Boards Parameters ###################################################
number_x_square: 3         # number of squares in the X direction
number_y_square: 4         # number of squares the Y direction

So it's my fault, I printed the board with wrong parameters.

Is it possible to add some warnings or error messages to prevent mistakes like this?

DawidStankiewicz commented 8 months ago

Leaving the trace, here is my correct (I guess) output cameras.yml:

%YAML:1.0
---
nb_camera: 1
camera_0:
   camera_matrix: !!opencv-matrix
      rows: 3
      cols: 3
      dt: d
      data: [ 3.4089219026871656e+03, 0., 1.9197051479126044e+03, 0.,
          3.4123589126351535e+03, 1.0866674255497603e+03, 0., 0., 1. ]
   distortion_vector: !!opencv-matrix
      rows: 1
      cols: 5
      dt: d
      data: [ -2.3354408055534709e-02, -1.8428057118907593e-02,
          3.5474013129843543e-04, 8.8942147821997972e-04,
          2.5178062482779479e-01 ]
   distortion_type: 0
   camera_group: 0
   img_width: 3840
   img_height: 2160
   camera_pose_matrix: !!opencv-matrix
      rows: 4
      cols: 4
      dt: d
      data: [ 1., 0., 0., 0., 0., 1., 0., 0., 0., 0., 1., 0., 0., 0., 0.,
          1. ]
errors:
   camera_0:
      translation_std: !!opencv-matrix
         rows: 3
         cols: 1
         dt: d
         data: [ 0., 0., 0. ]
      mean_rotation_error: 0.

using this configration

boards_index: []
cam_params_path: None
cam_prefix: Cam_
camera_params_file_name: cameras.yml
distortion_model: 0
distortion_per_camera: []
fix_intrinsic: 0
he_approach: 0
length_marker: 0.03
length_square: 0.04
min_perc_pts: 0.8
number_board: 1
number_camera: 1
number_iterations: 1000
number_x_square: 3
number_x_square_per_board: []
number_y_square: 4
number_y_square_per_board: []
ransac_threshold: 5
refine_corner: 1
resolution_x: 3840
resolution_y: 2160
save_detection: 1
save_reprojection: 1
square_size: 6.5
square_size_per_board: []

Example frame with detections:

000033

And here is full log of this calibration:

0000001 | 2023-11-09, 19:41:55.931742 [info] - Nb of cameras : 1   Nb of Boards : 1   Refined Corners : true   Distortion mode : 0
0000002 | 2023-11-09, 19:41:55.932443 [info] - Extraction camera 001
0000003 | 2023-11-09, 19:41:55.954703 [info] - Number of threads for board detection :: 20
0000154 | 2023-11-09, 19:41:56.391658 [info] - Number of detected corners :: 6
0000155 | 2023-11-09, 19:41:56.883855 [info] - Number of detected corners :: 6
0000156 | 2023-11-09, 19:41:57.014276 [info] - Number of detected corners :: 6
0000157 | 2023-11-09, 19:41:57.025579 [info] - Number of detected corners :: 6
0000158 | 2023-11-09, 19:41:57.035072 [info] - Number of detected corners :: 6
0000159 | 2023-11-09, 19:41:57.043034 [info] - Number of detected corners :: 6
0000160 | 2023-11-09, 19:41:57.046643 [info] - Number of detected corners :: 6
0000161 | 2023-11-09, 19:41:57.051452 [info] - Number of detected corners :: 6
0000162 | 2023-11-09, 19:41:57.052213 [info] - Number of detected corners :: 6
0000163 | 2023-11-09, 19:41:57.064255 [info] - Number of detected corners :: 6
0000164 | 2023-11-09, 19:41:57.067523 [info] - Number of detected corners :: 6
0000165 | 2023-11-09, 19:41:57.069675 [info] - Number of detected corners :: 6
0000166 | 2023-11-09, 19:41:57.071843 [info] - Number of detected corners :: 6
0000167 | 2023-11-09, 19:41:57.072352 [info] - Number of detected corners :: 6
0000168 | 2023-11-09, 19:41:57.074574 [info] - Number of detected corners :: 6
0000169 | 2023-11-09, 19:41:57.078415 [info] - Number of detected corners :: 6
0000170 | 2023-11-09, 19:41:57.078799 [info] - Number of detected corners :: 6
0000171 | 2023-11-09, 19:41:57.081711 [info] - Number of detected corners :: 6
0000172 | 2023-11-09, 19:41:57.187564 [info] - Number of detected corners :: 6
0000173 | 2023-11-09, 19:41:57.788288 [info] - Number of detected corners :: 6
0000174 | 2023-11-09, 19:41:57.846747 [info] - Number of detected corners :: 6
0000175 | 2023-11-09, 19:41:57.915139 [info] - Number of detected corners :: 6
0000176 | 2023-11-09, 19:41:58.377457 [info] - Number of detected corners :: 6
0000177 | 2023-11-09, 19:41:58.408988 [info] - Number of detected corners :: 6
0000178 | 2023-11-09, 19:41:58.417557 [info] - Number of detected corners :: 6
0000179 | 2023-11-09, 19:41:58.420223 [info] - Number of detected corners :: 6
0000180 | 2023-11-09, 19:41:58.422251 [info] - Number of detected corners :: 6
0000181 | 2023-11-09, 19:41:58.422873 [info] - Number of detected corners :: 6
0000182 | 2023-11-09, 19:41:58.424522 [info] - Number of detected corners :: 6
0000183 | 2023-11-09, 19:41:58.430306 [info] - Number of detected corners :: 6
0000184 | 2023-11-09, 19:41:58.435548 [info] - Number of detected corners :: 6
0000185 | 2023-11-09, 19:41:58.435657 [info] - Number of detected corners :: 6
0000186 | 2023-11-09, 19:41:58.439173 [info] - Number of detected corners :: 6
0000187 | 2023-11-09, 19:41:58.442691 [info] - Number of detected corners :: 6
0000188 | 2023-11-09, 19:41:58.443259 [info] - Number of detected corners :: 6
0000189 | 2023-11-09, 19:41:58.456990 [info] - Number of detected corners :: 6
0000190 | 2023-11-09, 19:41:58.468937 [info] - Number of detected corners :: 6
0000191 | 2023-11-09, 19:41:58.602380 [info] - Number of detected corners :: 6
0000192 | 2023-11-09, 19:41:59.179171 [info] - Number of detected corners :: 6
0000193 | 2023-11-09, 19:41:59.181499 [info] - Number of detected corners :: 6
0000194 | 2023-11-09, 19:41:59.230882 [info] - Number of detected corners :: 6
0000195 | 2023-11-09, 19:41:59.294396 [info] - Number of detected corners :: 6
0000196 | 2023-11-09, 19:41:59.616744 [info] - Number of detected corners :: 6
0000197 | 2023-11-09, 19:41:59.763195 [info] - Number of detected corners :: 6
0000198 | 2023-11-09, 19:41:59.772432 [info] - Number of detected corners :: 6
0000199 | 2023-11-09, 19:41:59.779965 [info] - Number of detected corners :: 6
0000200 | 2023-11-09, 19:41:59.791094 [info] - Number of detected corners :: 6
0000201 | 2023-11-09, 19:41:59.800951 [info] - Number of detected corners :: 6
0000202 | 2023-11-09, 19:41:59.804885 [info] - Number of detected corners :: 6
0000203 | 2023-11-09, 19:41:59.813097 [info] - Number of detected corners :: 6
0000204 | 2023-11-09, 19:41:59.820213 [info] - Number of detected corners :: 6
0000205 | 2023-11-09, 19:41:59.831461 [info] - Number of detected corners :: 6
0000206 | 2023-11-09, 19:41:59.856650 [info] - Number of detected corners :: 6
0000207 | 2023-11-09, 19:41:59.869923 [info] - Number of detected corners :: 6
0000208 | 2023-11-09, 19:41:59.985144 [info] - Number of detected corners :: 6
0000209 | 2023-11-09, 19:42:00.007011 [info] - Number of detected corners :: 6
0000210 | 2023-11-09, 19:42:00.028917 [info] - Number of detected corners :: 6
0000211 | 2023-11-09, 19:42:00.053372 [info] - Number of detected corners :: 6
0000212 | 2023-11-09, 19:42:00.190402 [info] - Number of detected corners :: 6
0000213 | 2023-11-09, 19:42:00.604648 [info] - Number of detected corners :: 6
0000214 | 2023-11-09, 19:42:00.696202 [info] - Number of detected corners :: 6
0000215 | 2023-11-09, 19:42:00.751165 [info] - Number of detected corners :: 6
0000216 | 2023-11-09, 19:42:00.817243 [info] - Number of detected corners :: 6
0000217 | 2023-11-09, 19:42:01.126580 [info] - Number of detected corners :: 6
0000218 | 2023-11-09, 19:42:01.163259 [info] - Number of detected corners :: 6
0000219 | 2023-11-09, 19:42:01.237139 [info] - Number of detected corners :: 6
0000220 | 2023-11-09, 19:42:01.263114 [info] - Number of detected corners :: 6
0000221 | 2023-11-09, 19:42:01.280151 [info] - Number of detected corners :: 6
0000222 | 2023-11-09, 19:42:01.286722 [info] - Number of detected corners :: 6
0000223 | 2023-11-09, 19:42:01.301338 [info] - Number of detected corners :: 6
0000224 | 2023-11-09, 19:42:01.305156 [info] - Number of detected corners :: 6
0000225 | 2023-11-09, 19:42:01.325084 [info] - Number of detected corners :: 6
0000226 | 2023-11-09, 19:42:01.484580 [info] - Number of detected corners :: 6
0000227 | 2023-11-09, 19:42:01.517241 [info] - Number of detected corners :: 6
0000228 | 2023-11-09, 19:42:01.528997 [info] - Number of detected corners :: 6
0000229 | 2023-11-09, 19:42:01.536519 [info] - Number of detected corners :: 6
0000230 | 2023-11-09, 19:42:01.541458 [info] - Number of detected corners :: 6
0000231 | 2023-11-09, 19:42:01.619411 [info] - Number of detected corners :: 6
0000232 | 2023-11-09, 19:42:02.040156 [info] - Number of detected corners :: 6
0000233 | 2023-11-09, 19:42:02.216112 [info] - Number of detected corners :: 6
0000234 | 2023-11-09, 19:42:02.234455 [info] - Number of detected corners :: 6
0000235 | 2023-11-09, 19:42:02.240154 [info] - Number of detected corners :: 6
0000236 | 2023-11-09, 19:42:02.256737 [info] - Number of detected corners :: 6
0000237 | 2023-11-09, 19:42:02.271854 [info] - Number of detected corners :: 6
0000238 | 2023-11-09, 19:42:02.380276 [info] - Number of detected corners :: 6
0000239 | 2023-11-09, 19:42:02.603583 [info] - Number of detected corners :: 6
0000240 | 2023-11-09, 19:42:02.612805 [info] - Number of detected corners :: 6
0000241 | 2023-11-09, 19:42:02.662214 [info] - Number of detected corners :: 6
0000242 | 2023-11-09, 19:42:02.701055 [info] - Number of detected corners :: 6
0000243 | 2023-11-09, 19:42:02.711006 [info] - Number of detected corners :: 6
0000244 | 2023-11-09, 19:42:02.737259 [info] - Number of detected corners :: 6
0000245 | 2023-11-09, 19:42:02.760583 [info] - Number of detected corners :: 6
0000246 | 2023-11-09, 19:42:02.950104 [info] - Number of detected corners :: 6
0000247 | 2023-11-09, 19:42:02.951831 [info] - Number of detected corners :: 6
0000248 | 2023-11-09, 19:42:02.967098 [info] - Number of detected corners :: 6
0000249 | 2023-11-09, 19:42:02.981049 [info] - Number of detected corners :: 6
0000250 | 2023-11-09, 19:42:03.077919 [info] - Number of detected corners :: 6
0000251 | 2023-11-09, 19:42:03.459420 [info] - Number of detected corners :: 6
0000252 | 2023-11-09, 19:42:03.689569 [info] - Number of detected corners :: 6
0000253 | 2023-11-09, 19:42:03.694179 [info] - Number of detected corners :: 6
0000254 | 2023-11-09, 19:42:03.704636 [info] - Number of detected corners :: 6
0000255 | 2023-11-09, 19:42:03.704853 [info] - Number of detected corners :: 6
0000256 | 2023-11-09, 19:42:03.705908 [info] - Number of detected corners :: 6
0000257 | 2023-11-09, 19:42:03.740245 [info] - Number of detected corners :: 6
0000258 | 2023-11-09, 19:42:03.753769 [info] - Number of detected corners :: 6
0000259 | 2023-11-09, 19:42:03.945341 [info] - Number of detected corners :: 6
0000260 | 2023-11-09, 19:42:04.002475 [info] - Number of detected corners :: 6
0000261 | 2023-11-09, 19:42:04.127464 [info] - Number of detected corners :: 6
0000262 | 2023-11-09, 19:42:04.153221 [info] - Number of detected corners :: 6
0000263 | 2023-11-09, 19:42:04.229629 [info] - Number of detected corners :: 6
0000264 | 2023-11-09, 19:42:04.284294 [info] - Number of detected corners :: 6
0000265 | 2023-11-09, 19:42:04.340753 [info] - Number of detected corners :: 6
0000266 | 2023-11-09, 19:42:04.530619 [info] - Number of detected corners :: 6
0000267 | 2023-11-09, 19:42:04.685421 [info] - Number of detected corners :: 6
0000268 | 2023-11-09, 19:42:04.897467 [info] - Number of detected corners :: 6
0000269 | 2023-11-09, 19:42:05.225240 [info] - Number of detected corners :: 6
0000270 | 2023-11-09, 19:42:05.255726 [info] - Number of detected corners :: 6
0000271 | 2023-11-09, 19:42:05.274892 [info] - Number of detected corners :: 6
0000272 | 2023-11-09, 19:42:05.276763 [info] - Number of detected corners :: 6
0000273 | 2023-11-09, 19:42:05.278098 [info] - Number of detected corners :: 6
0000274 | 2023-11-09, 19:42:05.286148 [info] - Number of detected corners :: 6
0000275 | 2023-11-09, 19:42:05.300776 [info] - Number of detected corners :: 6
0000276 | 2023-11-09, 19:42:05.372645 [info] - Number of detected corners :: 6
0000277 | 2023-11-09, 19:42:05.402864 [info] - Number of detected corners :: 6
0000278 | 2023-11-09, 19:42:05.423072 [info] - Number of detected corners :: 6
0000279 | 2023-11-09, 19:42:05.552719 [info] - Number of detected corners :: 6
0000280 | 2023-11-09, 19:42:05.555362 [info] - Number of detected corners :: 6
0000281 | 2023-11-09, 19:42:05.565536 [info] - Number of detected corners :: 6
0000282 | 2023-11-09, 19:42:05.713833 [info] - Number of detected corners :: 6
0000283 | 2023-11-09, 19:42:05.816718 [info] - Number of detected corners :: 6
0000284 | 2023-11-09, 19:42:06.029115 [info] - Board extraction done!
0000285 | 2023-11-09, 19:42:06.029148 [info] - Intrinsic calibration initiated
0000286 | 2023-11-09, 19:42:06.029156 [info] - Initializing camera calibration using images
0000287 | 2023-11-09, 19:42:06.029161 [info] - NB of board available in this camera :: 130
0000288 | 2023-11-09, 19:42:06.029172 [info] - NB of frames where this camera saw a board :: 130
0000289 | 2023-11-09, 19:42:18.670284 [info] - cameraMatrix : [3405.219129797035, 0, 1910.094988443297;
 0, 3410.201075447987, 1117.978734413974;
 0, 0, 1]
0000290 | 2023-11-09, 19:42:18.670343 [info] - distCoeffs : [0.03576338019176255, -0.3691663487185353, 0.002828214851890153, -0.0002464721235971383, 1.005100191367455]
0000681 | 2023-11-09, 19:42:18.695245 [info] - Parameters before optimization :: [3405.219129797035, 0, 1910.094988443297;
 0, 3410.201075447987, 1117.978734413974;
 0, 0, 1]
0000682 | 2023-11-09, 19:42:18.695289 [info] - distortion vector :: [0.03576338019176255, -0.3691663487185353, 0.002828214851890153, -0.0002464721235971383, 1.005100191367455]
iter      cost      cost_change  |gradient|   |step|    tr_ratio  tr_radius  ls_iter  iter_time  total_time
   0  7.290947e+01    0.00e+00    1.62e+03   0.00e+00   0.00e+00  1.00e+04        0    1.16e-02    1.26e-02
   1  6.926904e+01    3.64e+00    1.25e+03   3.05e+00   1.04e+00  3.00e+04        1    1.15e-02    2.41e-02
   2  6.782618e+01    1.44e+00    1.46e+02   1.64e+00   1.04e+00  9.00e+04        1    1.19e-02    3.61e-02
   3  6.735006e+01    4.76e-01    1.85e+02   3.37e+00   1.07e+00  2.70e+05        1    1.23e-02    4.84e-02
   4  6.707892e+01    2.71e-01    1.99e+02   6.53e+00   1.02e+00  8.10e+05        1    1.15e-02    5.99e-02
   5  6.680882e+01    2.70e-01    6.25e+02   1.06e+01   9.51e-01  2.43e+06        1    1.17e-02    7.17e-02
   6  6.668359e+01    1.25e-01    5.10e+02   9.81e+00   9.32e-01  6.82e+06        1    1.22e-02    8.39e-02
   7  6.666008e+01    2.35e-02    6.85e+01   3.64e+00   1.05e+00  2.05e+07        1    1.15e-02    9.54e-02
   8  6.665849e+01    1.59e-03    8.33e+00   5.95e-01   1.40e+00  6.14e+07        1    1.15e-02    1.07e-01
   9  6.665805e+01    4.35e-04    3.82e+00   1.03e-01   1.62e+00  1.84e+08        1    1.22e-02    1.19e-01
  10  6.665788e+01    1.72e-04    2.08e+00   3.96e-02   1.64e+00  5.53e+08        1    1.23e-02    1.31e-01
  11  6.665781e+01    7.05e-05    1.25e+00   2.13e-02   1.64e+00  1.66e+09        1    1.16e-02    1.43e-01
0000683 | 2023-11-09, 19:42:18.841165 [info] - Parameters after optimization :: [3408.921902687166, 0, 1919.705147912604;
 0, 3412.358912635154, 1086.66742554976;
 0, 0, 1]
0000684 | 2023-11-09, 19:42:18.841210 [info] - distortion vector after optimization :: [-0.02335440805553471, -0.01842805711890759, 0.0003547401312984354, 0.0008894214782199797, 0.2517806248277948]
0000685 | 2023-11-09, 19:42:18.843411 [info] - Intrinsic Calibration done!
0000686 | 2023-11-09, 19:42:18.843422 [info] - 3D Object calibration initiated
0001211 | 2023-11-09, 19:42:18.868771 [info] - Mean Error 0.351087
iter      cost      cost_change  |gradient|   |step|    tr_ratio  tr_radius  ls_iter  iter_time  total_time
   0  6.747345e+01    0.00e+00    3.39e+02   0.00e+00   0.00e+00  1.00e+04        0    7.18e-03    8.04e-03
   1  6.676715e+01    7.06e-01    2.11e+02   1.04e-01   1.34e+00  3.00e+04        1    8.07e-03    1.61e-02
   2  6.667512e+01    9.20e-02    3.11e+01   1.27e-02   1.17e+00  9.00e+04        1    8.78e-03    2.49e-02
   3  6.666652e+01    8.60e-03    2.49e+01   3.37e-03   1.60e+00  2.70e+05        1    7.91e-03    3.28e-02
   4  6.666276e+01    3.76e-03    2.00e+01   3.15e-03   1.72e+00  8.10e+05        1    7.81e-03    4.07e-02
   5  6.666071e+01    2.05e-03    1.61e+01   2.24e-03   1.77e+00  2.43e+06        1    7.82e-03    4.85e-02
   6  6.665944e+01    1.27e-03    1.30e+01   1.63e-03   1.81e+00  7.29e+06        1    8.64e-03    5.71e-02
   7  6.665867e+01    7.70e-04    1.05e+01   1.26e-03   1.63e+00  2.19e+07        1    7.87e-03    6.50e-02
   8  6.665830e+01    3.71e-04    8.49e+00   7.35e-04   1.74e+00  6.56e+07        1    7.80e-03    7.28e-02
   9  6.665809e+01    2.07e-04    6.88e+00   5.13e-04   1.76e+00  1.97e+08        1    7.94e-03    8.08e-02
  10  6.665797e+01    1.21e-04    5.58e+00   3.71e-04   1.78e+00  5.90e+08        1    8.74e-03    8.95e-02
  11  6.665789e+01    7.38e-05    4.53e+00   2.75e-04   1.79e+00  1.77e+09        1    8.35e-03    9.79e-02
0001342 | 2023-11-09, 19:42:18.972266 [info] - Mean Error 0.349113
0001343 | 2023-11-09, 19:42:18.972296 [info] - 3D Object calibration done!
0001344 | 2023-11-09, 19:42:18.972301 [info] - Camera group calibration initiated
0001348 | 2023-11-09, 19:42:18.975133 [info] - Number of frames for camera group optimization  :: 130
iter      cost      cost_change  |gradient|   |step|    tr_ratio  tr_radius  ls_iter  iter_time  total_time
   0  6.665789e+01    0.00e+00    4.53e+00   0.00e+00   0.00e+00  1.00e+04        0    1.06e-02    1.17e-02
0001349 | 2023-11-09, 19:42:18.990538 [info] - Camera  0  :: [1, 0, 0, 0;
 0, 1, 0, 0;
 0, 0, 1, 0;
 0, 0, 0, 1]
0001350 | 2023-11-09, 19:42:18.991763 [info] - Camera group calibration done!
0002262 | 2023-11-09, 19:42:19.048723 [info] - Number of frames for camera group optimization  :: 130
iter      cost      cost_change  |gradient|   |step|    tr_ratio  tr_radius  ls_iter  iter_time  total_time
   0  6.732523e+01    0.00e+00    3.39e+02   0.00e+00   0.00e+00  1.00e+04        0    9.89e-03    1.08e-02
   1  6.676684e+01    5.58e-01    2.11e+02   4.65e-02   1.46e+00  3.00e+04        1    1.18e-02    2.27e-02
   2  6.667512e+01    9.17e-02    3.11e+01   1.21e-02   1.17e+00  9.00e+04        1    1.11e-02    3.38e-02
   3  6.666652e+01    8.60e-03    2.49e+01   3.36e-03   1.60e+00  2.70e+05        1    1.17e-02    4.55e-02
   4  6.666276e+01    3.76e-03    2.00e+01   3.15e-03   1.72e+00  8.10e+05        1    1.09e-02    5.65e-02
   5  6.666071e+01    2.05e-03    1.61e+01   2.24e-03   1.77e+00  2.43e+06        1    1.09e-02    6.74e-02
   6  6.665944e+01    1.27e-03    1.30e+01   1.63e-03   1.81e+00  7.29e+06        1    1.16e-02    7.89e-02
   7  6.665867e+01    7.70e-04    1.05e+01   1.26e-03   1.63e+00  2.19e+07        1    1.10e-02    8.99e-02
   8  6.665830e+01    3.71e-04    8.49e+00   7.35e-04   1.74e+00  6.56e+07        1    1.17e-02    1.02e-01
   9  6.665809e+01    2.07e-04    6.88e+00   5.13e-04   1.76e+00  1.97e+08        1    1.09e-02    1.13e-01
  10  6.665797e+01    1.21e-04    5.58e+00   3.71e-04   1.78e+00  5.90e+08        1    1.15e-02    1.24e-01
  11  6.665789e+01    7.38e-05    4.53e+00   2.75e-04   1.79e+00  1.77e+09        1    1.09e-02    1.35e-01
0002263 | 2023-11-09, 19:42:19.187167 [info] - Camera  0  :: [1, 0, 0, 0;
 0, 1, 0, 0;
 0, 0, 1, 0;
 0, 0, 0, 1]
0002394 | 2023-11-09, 19:42:19.191057 [info] - Non-overlapping calibration initiated
0002395 | 2023-11-09, 19:42:19.191075 [info] - Non-overlapping calibration done!
0002396 | 2023-11-09, 19:42:19.191081 [info] - Merge cameras and objets initiated
0003308 | 2023-11-09, 19:42:19.246652 [info] - Number of frames for camera group optimization  :: 130
iter      cost      cost_change  |gradient|   |step|    tr_ratio  tr_radius  ls_iter  iter_time  total_time
   0  6.732523e+01    0.00e+00    3.39e+02   0.00e+00   0.00e+00  1.00e+04        0    9.97e-03    1.09e-02
   1  6.676684e+01    5.58e-01    2.11e+02   4.65e-02   1.46e+00  3.00e+04        1    1.12e-02    2.21e-02
   2  6.667512e+01    9.17e-02    3.11e+01   1.21e-02   1.17e+00  9.00e+04        1    1.18e-02    3.39e-02
   3  6.666652e+01    8.60e-03    2.49e+01   3.36e-03   1.60e+00  2.70e+05        1    1.17e-02    4.56e-02
   4  6.666276e+01    3.76e-03    2.00e+01   3.15e-03   1.72e+00  8.10e+05        1    1.10e-02    5.66e-02
   5  6.666071e+01    2.05e-03    1.61e+01   2.24e-03   1.77e+00  2.43e+06        1    1.16e-02    6.82e-02
   6  6.665944e+01    1.27e-03    1.30e+01   1.63e-03   1.81e+00  7.29e+06        1    1.09e-02    7.91e-02
   7  6.665867e+01    7.70e-04    1.05e+01   1.26e-03   1.63e+00  2.19e+07        1    1.08e-02    8.99e-02
   8  6.665830e+01    3.71e-04    8.49e+00   7.35e-04   1.74e+00  6.56e+07        1    1.72e-02    1.07e-01
   9  6.665809e+01    2.07e-04    6.88e+00   5.13e-04   1.76e+00  1.97e+08        1    1.08e-02    1.18e-01
  10  6.665797e+01    1.21e-04    5.58e+00   3.71e-04   1.78e+00  5.90e+08        1    1.16e-02    1.30e-01
  11  6.665789e+01    7.38e-05    4.53e+00   2.75e-04   1.79e+00  1.77e+09        1    1.09e-02    1.41e-01
0003309 | 2023-11-09, 19:42:19.390205 [info] - Camera  0  :: [1, 0, 0, 0;
 0, 1, 0, 0;
 0, 0, 1, 0;
 0, 0, 0, 1]
0003830 | 2023-11-09, 19:42:19.420297 [info] - Merge cameras and objets done!
0003831 | 2023-11-09, 19:42:19.420325 [info] - Final refinement initiated
0003832 | 2023-11-09, 19:42:19.420334 [info] - Number of frames for camera group optimization  :: 130
iter      cost      cost_change  |gradient|   |step|    tr_ratio  tr_radius  ls_iter  iter_time  total_time
   0  6.732523e+01    0.00e+00    3.39e+02   0.00e+00   0.00e+00  1.00e+04        0    9.94e-03    1.08e-02
   1  6.676684e+01    5.58e-01    2.11e+02   4.65e-02   1.46e+00  3.00e+04        1    1.17e-02    2.26e-02
   2  6.667512e+01    9.17e-02    3.11e+01   1.21e-02   1.17e+00  9.00e+04        1    1.09e-02    3.35e-02
   3  6.666652e+01    8.60e-03    2.49e+01   3.36e-03   1.60e+00  2.70e+05        1    1.08e-02    4.43e-02
   4  6.666276e+01    3.76e-03    2.00e+01   3.15e-03   1.72e+00  8.10e+05        1    1.15e-02    5.58e-02
   5  6.666071e+01    2.05e-03    1.61e+01   2.24e-03   1.77e+00  2.43e+06        1    1.08e-02    6.66e-02
   6  6.665944e+01    1.27e-03    1.30e+01   1.63e-03   1.81e+00  7.29e+06        1    1.08e-02    7.75e-02
   7  6.665867e+01    7.70e-04    1.05e+01   1.26e-03   1.63e+00  2.19e+07        1    1.15e-02    8.90e-02
   8  6.665830e+01    3.71e-04    8.49e+00   7.35e-04   1.74e+00  6.56e+07        1    1.08e-02    9.98e-02
   9  6.665809e+01    2.07e-04    6.88e+00   5.13e-04   1.76e+00  1.97e+08        1    1.07e-02    1.11e-01
  10  6.665797e+01    1.21e-04    5.58e+00   3.71e-04   1.78e+00  5.90e+08        1    1.17e-02    1.22e-01
  11  6.665789e+01    7.38e-05    4.53e+00   2.75e-04   1.79e+00  1.77e+09        1    1.08e-02    1.33e-01
0003833 | 2023-11-09, 19:42:19.556461 [info] - Camera  0  :: [1, 0, 0, 0;
 0, 1, 0, 0;
 0, 0, 1, 0;
 0, 0, 0, 1]
0003834 | 2023-11-09, 19:42:19.557965 [info] - Number of frames for camera group optimization  :: 130
iter      cost      cost_change  |gradient|   |step|    tr_ratio  tr_radius  ls_iter  iter_time  total_time
   0  6.665789e+01    0.00e+00    4.53e+00   0.00e+00   0.00e+00  1.00e+04        0    2.10e-02    2.21e-02
0003835 | 2023-11-09, 19:42:19.584015 [info] - Camera  0  :: [1, 0, 0, 0;
 0, 1, 0, 0;
 0, 0, 1, 0;
 0, 0, 0, 1]
0003966 | 2023-11-09, 19:42:19.587718 [info] - Final refinement done
0003967 | 2023-11-09, 19:42:36.831521 [info] - Save parameters
0003968 | 2023-11-09, 19:42:36.842377 [info] - mean reprojection error :: 0.349113
0003969 | 2023-11-09, 19:42:36.849276 [info] - Calibration took 40 seconds

=================================================================
==19==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 12312 byte(s) in 3 object(s) allocated from:
    #0 0x7f700471a787 in operator new[](unsigned long) ../../../../src/libsanitizer/asan/asan_new_delete.cc:107
    #1 0x7f700244066f  (/usr/lib/x86_64-linux-gnu/libtbb.so.2+0x2766f)

SUMMARY: AddressSanitizer: 12312 byte(s) leaked in 3 allocation(s).
rameau-fr commented 8 months ago

Thank you so much for the very detailed feedback; I am sure it will be useful for future users facing a similar issue. I am also glad you managed to find the mistake and calibrate your camera properly. The idea of warning the user that the used pattern does not exhibit the proper configuration is a great idea and might be implemented quite easily. Unfortunately, I cannot promise such an update rapidly. Thank you very much again!