vswdigitall commented 7 months ago

6x camera system calibration works before.

Now we build 4x camera system and the latest version of MCCalib failed.

System information (version)

Operating System / Platform => Ubuntu 22.04.3 64bit
OpenCV => 4.5.5
Ceres => 2.0.0
Boost => 1.74
C++ => c++17
Compiler => gcc11.4

Vision system

Number of cameras => 4
Types of cameras => fisheye
Multicamera configurations => non-overlapping
Configuration file => *.yml
Image sequences => 1495
- number of images per camera
- if relevant and possible, please share image sequences

Describe the issue / bug

After extracting keypoints always got this:

0009280 | 2023-11-28, 06:21:41.553512 [info] - Board extraction done! 0009281 | 2023-11-28, 06:21:41.553548 [info] - Intrinsic calibration initiated 0009282 | 2023-11-28, 06:21:41.553554 [info] - Initializing camera calibration using images 0009283 | 2023-11-28, 06:21:41.553559 [info] - NB of board available in this camera :: 678 0009284 | 2023-11-28, 06:21:41.553564 [info] - NB of frames where this camera saw a board :: 533 terminate called after throwing an instance of 'cv::Exception' what(): OpenCV(4.5.5) /home/vsw/opencv/opencv/modules/calib3d/src/fisheye.cpp:741: error: (-215:Assertion failed) !objectPoints.empty() && !imagePoints.empty() && objectPoints.total() == imagePoints.total() in function 'calibrate'

Please help.

vswdigitall commented 7 months ago

1170

rameau-fr commented 7 months ago

Thank you very much for your comment. According to the error you shared, it seems that the calibration failed due to the OpenCV's fisheye calibration function. This error occurs when no points have been detected for some cameras. Let's try to figure out what is falling:

Have you tried to calibrate the cameras individually to see if at least their intrinsic parameters can be estimated
Have you tested with the previous version of MC-Calib?
If possible, could you please share the configuration file and the entire log? Also, feel free to share some of your data with me so that I can run some tests.

Thank you again for using MC-Calib

vswdigitall commented 7 months ago

Will try calibrate every camera separately.

Calibrating with old stable version now. Will notice.

Config file:

%YAML:1.0

######################################## Print Boards Parameters ################################################### number_x_square: 6 #number of squares in the X direction number_y_square: 6 #number of squares the Y direction resolution_x: 2200 # horizontal resolution in pixel resolution_y: 2200 # vertical resolution in pixel length_square: 0.096 # parameters on the marker (can be kept as it is) length_marker: 0.072 # parameters on the marker (can be kept as it is) number_board: 6 # number of boards used for calibration (for overlapping camera 1 is enough ...) boards_index: [] #leave it empty [] if the board index are ranging from zero to number_board square_size: 0.128 # size of each square of the board in cm/mm/whatever you want

############# Boards Parameters for different board size (leave empty if all boards have the same size) ################# number_x_square_per_board: [] number_y_square_per_board: [] square_size_per_board: []

######################################## Camera Parameters ################################################### distortion_model: 1 #0:Brown (perspective) // 1: Kannala (fisheye) distortion_per_camera : [] #specify the model per camera,

leave "distortion_per_camera" empty [] if they all follow the same model (make sure that the vector is as long as cameras nb)

number_camera: 4 # number of cameras in the rig to calibrate refine_corner: 1 # activate or deactivate the corner refinement min_perc_pts: 0.5 # min percentage of points visible to assume a good detection

cam_params_path: "None" # "../../Images_Plan/calibrated_cameras_data.yml" # file with cameras intrinsics to initialize the intrinsic, write "None" if no initialization available

fix_intrinsic: 0 #if 1 then the intrinsic parameters will not be estimated nor refined (initial value needed)

######################################## Images Parameters ################################################### root_path: "/home/vsw/calib4/" cam_prefix: "cam" keypoints_path: "/home/vsw/calib4/detected_keypoints_data.yml"

######################################## Optimization Parameters ################################################### ransac_threshold: 50 #RANSAC threshold in pixel (keep it high just to remove strong outliers) ======50 number_iterations: 1000 #Max number of iterations for the non linear refinement quaternion_averaging: 1

######################################## Hand-eye method ############################################# he_approach: 0 #0: bootstrapped he technique, 1: traditional he =====1

######################################## Output Parameters ################################################### save_path: "/home/vsw/calib4/" save_detection: 1 save_reprojection: 1 camera_params_file_name: ""

vswdigitall commented 7 months ago

Old version of MCCalib same result.

Individual calibration of the first camera dataset failed.

Here is camera images: https://cam360.tv/etc/cam001.zip (2.7GB)

This is also needed: const float border_marging = 0.2f; // border margin tolerance

Please help.

rameau-fr commented 7 months ago

Thank you very much for sharing data with us, this is how we can make this tool better! The download of the images is quite slow, I will give a try as soon as I can

rameau-fr commented 7 months ago

For your configuration file, the part regarding the configuration of the calibration boards is missing, could you please provide it to me?

I could replicate your error, I strongly suspect a problem in the boards generation, I will wait for the missing part of the configuration file to be more conclusive on this regards.

vswdigitall commented 7 months ago

Have shared full config.

rameau-fr commented 7 months ago

Thanks a lot, so here is the problem. The boards you are using do not correspond to the boards generated from this configuration. I have identified the symbols, and they are not arranged in the same order. I am afraid you will have to reprint the calibration patterns: charuco_board_000 May I ask you how these patterns have been generated? Were you using the same calibration boards in your previous experiments? I do not think the board generation has been modified with the novel updates of the toolbox, so it is surprising to me (or it could be due to an update from OpenCV, difficult to say). The good news is that I believe that you should be able to make it work if you use the proper boards generated by MC-Calib using the apps/create_charuco_boards. Please keep us in touch

vswdigitall commented 7 months ago

Boards were generated with your tool, here is same boards setup and calibrated successful 6 camera rig.

vswdigitall commented 7 months ago

Here they are: https://cam360.tv/etc/charuco_board.zip

We works with them much time.

vswdigitall commented 7 months ago

Here is config for generating:

%YAML:1.0

number_x_square: 6 #number of squares in the X direction number_y_square: 6
resolution_x: 2200 # 747mm resolution_y: 2200
length_square: 0.04 # no need to modify!! length_marker: 0.03 # no need to modify!! number_board: 6 boards_index: [ ] square_size: 0.12 # the size of each square in the real world

vswdigitall commented 7 months ago

Looks like your tool was changed.

rameau-fr commented 7 months ago

I compared your board with the one generated from MC-Calib and I can confirm they are different. However, I do not understand why. I noticed that the board you use exactly corresponds to the one generated from calib.io which are indeed incompatible with our toolbox.

vswdigitall commented 7 months ago

It is amazing. Have generated and cells were shifted. But it worked before. Will test old dataset and reply.

vswdigitall commented 7 months ago

Meanwhile you can see our old conversation with these boards at vswdigitall commented on Sep 3, 2022

vswdigitall commented 7 months ago

May be it depends on opencv version?

rameau-fr commented 7 months ago

I was also looking at your previous configuration, and indeed, the patterns were precisely the same, and the code was working well. Back in the day, we were maybe using another Docker with another version of OpenCV, it is rather difficult for me to recall for sure. I will try with our previous Docker version to see if it is related to the OpenCV version and I will keep you updated.

rameau-fr commented 7 months ago

Even with an older version of the code and a different Docker, I couldn't generate the same patterns as the ones you're currently using. You might try printing a newly generated pattern on an A4 paper and calibrating a single camera to see if it works. Honestly, it's quite a mystery to me, and I'm very curious about what happened to your configuration. Please keep us updated on future developments!

vswdigitall commented 7 months ago

Just calibrated old dataset with your latest software:

0003101 | 2023-11-28, 19:20:52.098933 [info] - Number of detected corners :: 23 0003102 | 2023-11-28, 19:20:52.468214 [info] - Number of detected corners :: 25 0003103 | 2023-11-28, 19:20:53.146892 [info] - Number of detected corners :: 25 0003104 | 2023-11-28, 19:20:53.646522 [info] - Number of detected corners :: 19 0003105 | 2023-11-28, 19:20:54.211584 [info] - Number of detected corners :: 25 0003106 | 2023-11-28, 19:20:54.260276 [info] - Number of detected corners :: 21 0003107 | 2023-11-28, 19:20:54.676375 [info] - Number of detected corners :: 23 0003108 | 2023-11-28, 19:20:55.320299 [info] - Number of detected corners :: 25 0003109 | 2023-11-28, 19:20:55.765710 [info] - Number of detected corners :: 23 0003110 | 2023-11-28, 19:20:57.366830 [info] - Number of detected corners :: 23 0003111 | 2023-11-28, 19:20:57.717597 [info] - Number of detected corners :: 21 0003112 | 2023-11-28, 19:20:58.325234 [info] - Number of detected corners :: 25 0003113 | 2023-11-28, 19:20:58.329851 [info] - Number of detected corners :: 25 0003114 | 2023-11-28, 19:20:58.992975 [info] - Number of detected corners :: 23 0003115 | 2023-11-28, 19:20:59.335424 [info] - Number of detected corners :: 25 0003116 | 2023-11-28, 19:20:59.440956 [info] - Number of detected corners :: 25 0003117 | 2023-11-28, 19:21:00.772678 [info] - Number of detected corners :: 25 0003118 | 2023-11-28, 19:21:01.793892 [info] - Number of detected corners :: 22 0003119 | 2023-11-28, 19:21:01.957634 [info] - Number of detected corners :: 25 0003120 | 2023-11-28, 19:21:02.067782 [info] - Number of detected corners :: 23 0003121 | 2023-11-28, 19:21:03.025754 [info] - Number of detected corners :: 25 0003122 | 2023-11-28, 19:21:03.514453 [info] - Number of detected corners :: 25 0003123 | 2023-11-28, 19:21:05.297043 [info] - Number of detected corners :: 20 0003124 | 2023-11-28, 19:21:05.541042 [info] - Number of detected corners :: 25 0003125 | 2023-11-28, 19:21:06.629728 [info] - Number of detected corners :: 21 0003126 | 2023-11-28, 19:21:10.040753 [info] - Number of detected corners :: 21 0003127 | 2023-11-28, 19:22:46.360134 [info] - Number of detected corners :: 25 0003128 | 2023-11-28, 19:23:10.685443 [info] - Number of detected corners :: 19 0003129 | 2023-11-28, 19:23:10.777603 [info] - Number of detected corners :: 25 0003130 | 2023-11-28, 19:23:12.253803 [info] - Number of detected corners :: 23 0003131 | 2023-11-28, 19:23:13.392775 [info] - Number of detected corners :: 25 0003132 | 2023-11-28, 19:23:13.650384 [info] - Number of detected corners :: 21 0003133 | 2023-11-28, 19:23:14.730178 [info] - Number of detected corners :: 23 0003134 | 2023-11-28, 19:23:15.250050 [info] - Number of detected corners :: 25 0003135 | 2023-11-28, 19:23:16.342283 [info] - Number of detected corners :: 25 0003136 | 2023-11-28, 19:23:16.618651 [info] - Number of detected corners :: 21 0003137 | 2023-11-28, 19:23:17.134627 [info] - Number of detected corners :: 19 0003138 | 2023-11-28, 19:23:18.214253 [info] - Number of detected corners :: 25 0003139 | 2023-11-28, 19:23:18.970185 [info] - Number of detected corners :: 21 0003140 | 2023-11-28, 19:23:19.216223 [info] - Number of detected corners :: 22 0003141 | 2023-11-28, 19:23:19.293189 [info] - Number of detected corners :: 25 0003142 | 2023-11-28, 19:23:20.189763 [info] - Number of detected corners :: 23 0003143 | 2023-11-28, 19:23:20.332085 [info] - Number of detected corners :: 25 0003144 | 2023-11-28, 19:23:20.862151 [info] - Number of detected corners :: 19 0003145 | 2023-11-28, 19:23:21.986806 [info] - Number of detected corners :: 23 0003146 | 2023-11-28, 19:23:23.334615 [info] - Number of detected corners :: 23 0003147 | 2023-11-28, 19:23:23.504795 [info] - Number of detected corners :: 23 0003148 | 2023-11-28, 19:23:23.786378 [info] - Number of detected corners :: 23 0003149 | 2023-11-28, 19:23:24.055176 [info] - Number of detected corners :: 24 0003150 | 2023-11-28, 19:23:25.224116 [info] - Number of detected corners :: 25 0003151 | 2023-11-28, 19:23:25.936353 [info] - Number of detected corners :: 23 0003152 | 2023-11-28, 19:23:26.337994 [info] - Number of detected corners :: 25 0003153 | 2023-11-28, 19:23:26.980438 [info] - Number of detected corners :: 22 0003154 | 2023-11-28, 19:23:27.049238 [info] - Number of detected corners :: 23 0003155 | 2023-11-28, 19:23:27.172439 [info] - Number of detected corners :: 23 0003156 | 2023-11-28, 19:23:27.263027 [info] - Number of detected corners :: 25 0003157 | 2023-11-28, 19:23:27.374671 [info] - Number of detected corners :: 25 0003158 | 2023-11-28, 19:23:28.033547 [info] - Number of detected corners :: 25 0003159 | 2023-11-28, 19:23:29.685472 [info] - Number of detected corners :: 25 0003160 | 2023-11-28, 19:23:30.480809 [info] - Number of detected corners :: 25 0003161 | 2023-11-28, 19:23:30.571279 [info] - Number of detected corners :: 25 0003162 | 2023-11-28, 19:23:30.604656 [info] - Number of detected corners :: 25 0003163 | 2023-11-28, 19:23:31.236201 [info] - Number of detected corners :: 25 0003164 | 2023-11-28, 19:23:31.529349 [info] - Number of detected corners :: 23 0003165 | 2023-11-28, 19:23:33.228047 [info] - Number of detected corners :: 25 0003166 | 2023-11-28, 19:23:33.978810 [info] - Number of detected corners :: 25 0003167 | 2023-11-28, 19:23:34.111928 [info] - Number of detected corners :: 21 0003168 | 2023-11-28, 19:23:35.101707 [info] - Number of detected corners :: 25 0003169 | 2023-11-28, 19:23:35.113195 [info] - Number of detected corners :: 25 0003170 | 2023-11-28, 19:23:35.504476 [info] - Number of detected corners :: 25 0003171 | 2023-11-28, 19:23:37.136754 [info] - Number of detected corners :: 25 0003172 | 2023-11-28, 19:23:38.494691 [info] - Number of detected corners :: 23 0003173 | 2023-11-28, 19:23:39.220065 [info] - Number of detected corners :: 25 0003174 | 2023-11-28, 19:23:39.303539 [info] - Number of detected corners :: 21 0003175 | 2023-11-28, 19:23:40.850767 [info] - Number of detected corners :: 21 0003176 | 2023-11-28, 19:23:43.180925 [info] - Number of detected corners :: 25 0003177 | 2023-11-28, 19:25:41.308406 [info] - Number of detected corners :: 25 0003178 | 2023-11-28, 19:25:41.318123 [info] - Number of detected corners :: 21 0003179 | 2023-11-28, 19:25:44.215929 [info] - Number of detected corners :: 21 0003180 | 2023-11-28, 19:25:44.803225 [info] - Number of detected corners :: 25 0003181 | 2023-11-28, 19:25:45.401824 [info] - Number of detected corners :: 25 0003182 | 2023-11-28, 19:25:45.670900 [info] - Number of detected corners :: 25 0003183 | 2023-11-28, 19:25:45.938914 [info] - Number of detected corners :: 23 0003184 | 2023-11-28, 19:25:47.330062 [info] - Number of detected corners :: 25 0003185 | 2023-11-28, 19:25:48.410660 [info] - Number of detected corners :: 25 0003186 | 2023-11-28, 19:25:49.450858 [info] - Number of detected corners :: 25 0003187 | 2023-11-28, 19:25:49.748796 [info] - Number of detected corners :: 25 0003188 | 2023-11-28, 19:25:50.493401 [info] - Number of detected corners :: 24 0003189 | 2023-11-28, 19:25:51.416424 [info] - Number of detected corners :: 21 0003190 | 2023-11-28, 19:25:51.990358 [info] - Number of detected corners :: 21 0003191 | 2023-11-28, 19:25:53.154336 [info] - Number of detected corners :: 25 0003192 | 2023-11-28, 19:25:53.383648 [info] - Number of detected corners :: 25 0003193 | 2023-11-28, 19:25:53.506299 [info] - Number of detected corners :: 19 0003194 | 2023-11-28, 19:25:55.138483 [info] - Number of detected corners :: 25 0003195 | 2023-11-28, 19:25:55.750097 [info] - Number of detected corners :: 25 0003196 | 2023-11-28, 19:25:57.257108 [info] - Number of detected corners :: 25 0003197 | 2023-11-28, 19:25:58.588317 [info] - Number of detected corners :: 21 0003198 | 2023-11-28, 19:25:58.864640 [info] - Number of detected corners :: 25 0003199 | 2023-11-28, 19:25:59.354665 [info] - Number of detected corners :: 23 0003200 | 2023-11-28, 19:26:00.348923 [info] - Number of detected corners :: 23 0003201 | 2023-11-28, 19:26:00.635875 [info] - Number of detected corners :: 25 0003202 | 2023-11-28, 19:26:02.110292 [info] - Number of detected corners :: 23 0003203 | 2023-11-28, 19:26:02.306138 [info] - Number of detected corners :: 23 0003204 | 2023-11-28, 19:26:02.502531 [info] - Number of detected corners :: 25 0003205 | 2023-11-28, 19:26:02.894201 [info] - Number of detected corners :: 25 0003206 | 2023-11-28, 19:26:03.740079 [info] - Number of detected corners :: 23 0003207 | 2023-11-28, 19:26:03.836921 [info] - Number of detected corners :: 25 0003208 | 2023-11-28, 19:26:04.241947 [info] - Number of detected corners :: 25 0003209 | 2023-11-28, 19:26:05.301464 [info] - Number of detected corners :: 25 0003210 | 2023-11-28, 19:26:05.769788 [info] - Number of detected corners :: 21 0003211 | 2023-11-28, 19:26:05.852684 [info] - Number of detected corners :: 25 0003212 | 2023-11-28, 19:26:06.862606 [info] - Number of detected corners :: 25 0003213 | 2023-11-28, 19:26:07.278299 [info] - Number of detected corners :: 25 0003214 | 2023-11-28, 19:26:08.333024 [info] - Number of detected corners :: 23 0003215 | 2023-11-28, 19:26:08.813410 [info] - Number of detected corners :: 25 0003216 | 2023-11-28, 19:26:09.518559 [info] - Number of detected corners :: 25 0003217 | 2023-11-28, 19:26:10.324755 [info] - Number of detected corners :: 21 0003218 | 2023-11-28, 19:26:11.726350 [info] - Number of detected corners :: 25 0003219 | 2023-11-28, 19:26:12.900105 [info] - Number of detected corners :: 25 0003220 | 2023-11-28, 19:26:14.666092 [info] - Number of detected corners :: 23 0003221 | 2023-11-28, 19:26:14.683859 [info] - Number of detected corners :: 19 0003222 | 2023-11-28, 19:26:15.509954 [info] - Number of detected corners :: 25 0003223 | 2023-11-28, 19:26:18.989951 [info] - Number of detected corners :: 19 0003224 | 2023-11-28, 19:26:46.814858 [info] - Board extraction done! 0003225 | 2023-11-28, 19:26:46.814899 [info] - Intrinsic calibration initiated 0003226 | 2023-11-28, 19:26:46.814905 [info] - Initializing camera calibration using images 0003227 | 2023-11-28, 19:26:46.814909 [info] - NB of board available in this camera :: 1112 0003228 | 2023-11-28, 19:26:46.814913 [info] - NB of frames where this camera saw a board :: 497 0003229 | 2023-11-28, 19:26:48.045180 [info] - cameraMatrix : [1750.890810413731, 0, 2027.760762863973; 0, 1750.694019479435, 1488.007897822958; 0, 0, 1] 0003230 | 2023-11-28, 19:26:48.045887 [info] - distCoeffs : [0.05428445118528479; 0.002450981754032617; 0.01167918960434057; -0.009365215104920499] 0006567 | 2023-11-28, 19:26:48.241306 [info] - Parameters before optimization :: [1750.890810413731, 0, 2027.760762863973; 0, 1750.694019479435, 1488.007897822958; 0, 0, 1] 0006568 | 2023-11-28, 19:26:48.241362 [info] - distortion vector :: [0.05428445118528479, 0.002450981754032617, -0.009365215104920499, 0] iter cost cost_change |gradient| |step| tr_ratio tr_radius ls_iter iter_time total_time 0 2.734201e+04 0.00e+00 2.53e+06 0.00e+00 0.00e+00 1.00e+04 0 3.91e-02 6.61e-02 1 9.403841e+03 1.79e+04 2.44e+06 1.14e+01 1.16e+00 3.00e+04 1 5.16e-02 1.18e-01 2 2.563567e+03 6.84e+03 7.87e+05 7.13e+00 1.42e+00 9.00e+04 1 4.94e-02 1.68e-01 3 2.562308e+03 1.26e+00 2.02e+04 6.63e+00 3.64e-03 4.55e+04 1 4.94e-02 2.17e-01 4 1.824992e+03 7.37e+02 5.32e+04 1.33e+00 1.67e+00 1.36e+05 1 5.09e-02 2.68e-01 5 1.633598e+03 1.91e+02 5.61e+03 8.90e-01 1.53e+00 4.09e+05 1 5.10e-02 3.19e-01 6 1.617330e+03 1.63e+01 1.87e+03 4.79e-01 1.33e+00 1.23e+06 1 4.05e-01 7.24e-01 7 1.615679e+03 1.65e+00 2.03e+03 1.56e-01 1.81e+00 3.68e+06 1 4.58e-02 7.70e-01 8 1.614312e+03 1.37e+00 2.13e+03 5.77e-02 1.95e+00 1.11e+07 1 4.50e-02 8.15e-01 9 1.612872e+03 1.44e+00 1.64e+03 4.78e-02 2.07e+00 3.32e+07 1 4.51e-02 8.60e-01 10 1.611362e+03 1.51e+00 3.42e+02 4.82e-02 2.00e+00 9.95e+07 1 4.58e-02 9.06e-01 11 1.609369e+03 1.99e+00 3.75e+02 5.34e-02 2.15e+00 2.98e+08 1 4.61e-02 9.52e-01 12 1.605374e+03 3.99e+00 1.04e+03 7.60e-02 2.44e+00 8.95e+08 1 4.90e-02 1.00e+00 13 1.588649e+03 1.67e+01 5.48e+03 1.57e-01 2.60e+00 2.69e+09 1 4.80e-02 1.05e+00 14 1.713204e+03 -1.25e+02 5.48e+03 4.86e-01 -2.90e+00 1.34e+09 1 1.55e-02 1.06e+00 15 1.713204e+03 -1.25e+02 5.48e+03 4.86e-01 -2.90e+00 3.36e+08 1 1.39e-02 1.08e+00 16 1.713203e+03 -1.25e+02 5.48e+03 4.86e-01 -2.90e+00 4.20e+07 1 1.42e-02 1.09e+00 17 1.713197e+03 -1.25e+02 5.48e+03 4.85e-01 -2.90e+00 2.62e+06 1 1.41e-02 1.11e+00 18 1.713092e+03 -1.24e+02 5.48e+03 4.79e-01 -2.90e+00 8.20e+04 1 1.41e-02 1.12e+00 19 1.709915e+03 -1.21e+02 5.48e+03 4.06e-01 -2.83e+00 1.28e+03 1 1.42e-02 1.13e+00 20 1.614551e+03 -2.59e+01 5.48e+03 2.94e-01 -6.36e-01 1.00e+01 1 1.41e-02 1.15e+00 21 1.576008e+03 1.26e+01 2.36e+03 2.97e-02 1.18e+00 3.00e+01 1 4.85e-02 1.20e+00 22 1.558007e+03 1.80e+01 1.95e+03 6.33e-02 1.26e+00 9.01e+01 1 4.91e-02 1.25e+00 23 1.534559e+03 2.34e+01 3.20e+03 8.38e-02 1.31e+00 2.70e+02 1 4.90e-02 1.30e+00 24 1.515624e+03 1.89e+01 3.97e+03 6.78e-02 1.20e+00 8.11e+02 1 6.11e-02 1.36e+00 25 1.495617e+03 2.00e+01 6.50e+03 7.51e-02 1.30e+00 2.43e+03 1 4.90e-02 1.41e+00 26 1.488350e+03 7.27e+00 2.36e+03 3.70e-02 1.16e+00 7.30e+03 1 4.92e-02 1.45e+00 27 1.487797e+03 5.52e-01 1.83e+02 3.67e-02 1.59e+00 2.19e+04 1 6.23e-02 1.52e+00 28 1.487208e+03 5.89e-01 2.49e+02 6.96e-02 2.12e+00 6.57e+04 1 6.10e-02 1.58e+00 29 1.486293e+03 9.16e-01 3.47e+02 9.88e-02 2.20e+00 1.97e+05 1 4.95e-02 1.63e+00 30 1.484788e+03 1.50e+00 4.29e+02 9.00e-02 2.26e+00 5.91e+05 1 5.08e-02 1.68e+00 31 1.482038e+03 2.75e+00 6.54e+02 6.13e-02 2.36e+00 1.77e+06 1 5.09e-02 1.73e+00 32 1.476520e+03 5.52e+00 6.58e+02 9.16e-02 2.36e+00 5.32e+06 1 5.05e-02 1.78e+00 33 1.458940e+03 1.76e+01 2.04e+03 2.00e-01 2.67e+00 1.60e+07 1 6.21e-02 1.84e+00 34 1.556814e+03 -9.79e+01 2.04e+03 4.67e-01 -2.27e+00 7.98e+06 1 1.60e-02 1.86e+00 35 1.556795e+03 -9.79e+01 2.04e+03 4.66e-01 -2.27e+00 1.99e+06 1 2.69e-02 1.89e+00 36 1.556683e+03 -9.77e+01 2.04e+03 4.60e-01 -2.26e+00 2.49e+05 1 1.50e-02 1.90e+00 37 1.555635e+03 -9.67e+01 2.04e+03 4.23e-01 -2.24e+00 1.56e+04 1 1.51e-02 1.92e+00 38 1.538539e+03 -7.96e+01 2.04e+03 3.60e-01 -1.85e+00 4.87e+02 1 1.50e-02 1.93e+00 39 1.417344e+03 4.16e+01 6.66e+03 1.70e-01 1.39e+00 1.46e+03 1 4.97e-02 1.98e+00 40 1.479027e+03 -6.17e+01 6.66e+03 3.49e-01 -6.16e-01 7.30e+02 1 1.59e-02 2.00e+00 41 1.423717e+03 -6.37e+00 6.66e+03 2.99e-01 -6.73e-02 1.83e+02 1 2.61e-02 2.02e+00 42 1.346506e+03 7.08e+01 1.17e+04 1.73e-01 1.03e+00 5.48e+02 1 6.20e-02 2.08e+00 43 1.237906e+03 1.09e+02 1.60e+04 1.58e-01 1.29e+00 1.64e+03 1 6.18e-02 2.15e+00 44 1.166869e+03 7.10e+01 7.53e+03 1.48e-01 1.64e+00 4.93e+03 1 6.23e-02 2.21e+00 45 1.240941e+03 -7.41e+01 7.53e+03 2.97e-01 -1.18e+00 2.46e+03 1 1.60e-02 2.22e+00 46 1.218182e+03 -5.13e+01 7.53e+03 2.75e-01 -8.27e-01 6.16e+02 1 2.68e-02 2.25e+00 47 1.150532e+03 1.63e+01 1.80e+04 1.95e-01 3.01e-01 5.80e+02 1 5.99e-02 2.31e+00 48 1.013529e+03 1.37e+02 7.04e+03 1.13e-01 1.83e+00 1.74e+03 1 6.18e-02 2.37e+00 49 1.101004e+03 -8.75e+01 7.04e+03 3.00e-01 -1.74e+00 8.70e+02 1 1.55e-02 2.39e+00 50 1.046517e+03 -3.30e+01 7.04e+03 2.53e-01 -7.06e-01 2.17e+02 1 2.17e-02 2.41e+00 51 9.757744e+02 3.78e+01 9.26e+03 1.35e-01 1.23e+00 6.52e+02 1 6.48e-02 2.48e+00 52 9.341862e+02 4.16e+01 1.56e+04 1.57e-01 7.85e-01 8.00e+02 1 6.08e-02 2.54e+00 53 8.751437e+02 5.90e+01 3.97e+02 1.38e-02 1.67e+00 2.40e+03 1 5.99e-02 2.60e+00 54 8.751123e+02 3.14e-02 2.27e+01 2.94e-02 1.00e+00 7.20e+03 1 4.86e-02 2.64e+00 55 8.750726e+02 3.97e-02 2.79e+01 8.44e-02 1.00e+00 2.16e+04 1 4.92e-02 2.69e+00 56 8.749902e+02 8.24e-02 9.07e+01 1.96e-01 1.00e+00 6.48e+04 1 6.09e-02 2.75e+00 57 8.748843e+02 1.06e-01 2.60e+02 3.40e-01 9.99e-01 1.94e+05 1 6.18e-02 2.82e+00 58 8.748289e+02 5.55e-02 2.75e+02 3.46e-01 9.99e-01 5.83e+05 1 6.05e-02 2.88e+00 59 8.748214e+02 7.45e-03 7.38e+01 1.56e-01 1.00e+00 1.75e+06 1 6.07e-02 2.94e+00 0006569 | 2023-11-28, 19:26:51.212607 [info] - Parameters after optimization :: [1754.019321004974, 0, 2026.923497681134; 0, 1753.992572438344, 1486.29259742717; 0, 0, 1] 0006570 | 2023-11-28, 19:26:51.212662 [info] - distortion vector after optimization :: [0.05235148609092978, 0.01105175199417399, -0.00112027191927817, 0] 0006571 | 2023-11-28, 19:26:51.243067 [info] - Intrinsic Calibration done! 0006572 | 2023-11-28, 19:26:51.243097 [info] - 3D Object calibration initiated 0010744 | 2023-11-28, 19:26:51.366936 [info] - Mean Error 0.795021 iter cost cost_change |gradient| |step| tr_ratio tr_radius ls_iter iter_time total_time 0 2.415042e+04 0.00e+00 8.22e+05 0.00e+00 0.00e+00 1.00e+04 0 7.88e-02 8.95e-02 1 1.354273e+03 2.28e+04 1.24e+05 5.11e-01 1.54e+00 3.00e+04 1 4.10e-02 1.31e-01 2 1.113616e+03 2.41e+02 5.43e+03 6.96e-02 1.11e+00 9.00e+04 1 4.10e-02 1.72e-01 3 1.112761e+03 8.55e-01 1.66e+02 7.12e-03 1.01e+00 2.70e+05 1 4.12e-02 2.13e-01 0011242 | 2023-11-28, 19:26:51.637047 [info] - Mean Error 1.19725 0011243 | 2023-11-28, 19:26:51.637080 [info] - 3D Object calibration done! 0011244 | 2023-11-28, 19:26:51.637083 [info] - Camera group calibration initiated 0011248 | 2023-11-28, 19:26:51.644644 [info] - Number of frames for camera group optimization :: 497 iter cost cost_change |gradient| |step| tr_ratio tr_radius ls_iter iter_time total_time 0 1.112761e+03 0.00e+00 1.66e+02 0.00e+00 0.00e+00 1.00e+04 0 5.65e-02 8.20e-02 0011249 | 2023-11-28, 19:26:51.760974 [info] - Camera 0 :: [1, 0, 0, 0; 0, 1, 0, 0; 0, 0, 1, 0; 0, 0, 0, 1] 0011250 | 2023-11-28, 19:26:51.775978 [info] - Camera group calibration done! 0014731 | 2023-11-28, 19:26:51.997213 [info] - Number of frames for camera group optimization :: 497 iter cost cost_change |gradient| |step| tr_ratio tr_radius ls_iter iter_time total_time 0 2.401565e+04 0.00e+00 5.73e+05 0.00e+00 0.00e+00 1.00e+04 0 6.96e-02 8.30e-02 1 1.349890e+03 2.27e+04 1.28e+05 5.07e-01 1.54e+00 3.00e+04 1 8.86e-02 1.72e-01 2 1.113635e+03 2.36e+02 5.63e+03 6.98e-02 1.11e+00 9.00e+04 1 8.78e-02 2.59e-01 3 1.112761e+03 8.74e-01 1.76e+02 7.21e-03 1.01e+00 2.70e+05 1 8.60e-02 3.45e-01 0014732 | 2023-11-28, 19:26:52.377217 [info] - Camera 0 :: [1, 0, 0, 0; 0, 1, 0, 0; 0, 0, 1, 0; 0, 0, 0, 1] 0015230 | 2023-11-28, 19:26:52.407574 [info] - Non-overlapping calibration initiated 0015231 | 2023-11-28, 19:26:52.407601 [info] - Non-overlapping calibration done! 0015232 | 2023-11-28, 19:26:52.407604 [info] - Merge cameras and objets initiated 0018713 | 2023-11-28, 19:26:52.625750 [info] - Number of frames for camera group optimization :: 497 iter cost cost_change |gradient| |step| tr_ratio tr_radius ls_iter iter_time total_time 0 2.446264e+04 0.00e+00 5.71e+05 0.00e+00 0.00e+00 1.00e+04 0 5.86e-02 7.16e-02 1 1.722023e+03 2.27e+04 1.27e+05 5.16e-01 1.54e+00 3.00e+04 1 8.87e-02 1.60e-01 2 1.459241e+03 2.63e+02 6.87e+03 8.07e-02 1.11e+00 9.00e+04 1 7.58e-02 2.36e-01 3 1.457546e+03 1.69e+00 1.00e+03 2.49e-02 1.14e+00 2.70e+05 1 8.70e-02 3.23e-01 4 1.457276e+03 2.70e-01 5.57e+02 1.17e-02 1.45e+00 8.10e+05 1 7.95e-02 4.03e-01 5 1.457188e+03 8.79e-02 3.08e+02 6.03e-03 1.54e+00 2.43e+06 1 9.15e-02 4.94e-01 6 1.457155e+03 3.31e-02 1.75e+02 3.68e-03 1.62e+00 7.29e+06 1 8.92e-02 5.83e-01 7 1.457140e+03 1.49e-02 1.02e+02 2.69e-03 1.69e+00 2.19e+07 1 8.94e-02 6.73e-01 8 1.457132e+03 7.89e-03 6.04e+01 2.18e-03 1.74e+00 6.56e+07 1 7.77e-02 7.51e-01 9 1.457127e+03 4.71e-03 3.63e+01 1.86e-03 1.79e+00 1.97e+08 1 7.79e-02 8.28e-01 10 1.457124e+03 3.07e-03 2.56e+01 1.61e-03 1.82e+00 5.90e+08 1 9.00e-02 9.18e-01 11 1.457122e+03 2.13e-03 2.04e+01 1.41e-03 1.84e+00 1.77e+09 1 8.97e-02 1.01e+00 12 1.457121e+03 1.54e-03 1.66e+01 1.23e-03 1.86e+00 5.31e+09 1 8.96e-02 1.10e+00 0018714 | 2023-11-28, 19:26:53.758290 [info] - Camera 0 :: [1, 0, 0, 0; 0, 1, 0, 0; 0, 0, 1, 0; 0, 0, 0, 1] 0020703 | 2023-11-28, 19:26:53.898122 [info] - Merge cameras and objets done! 0020704 | 2023-11-28, 19:26:53.898152 [info] - Final refinement initiated 0020705 | 2023-11-28, 19:26:53.898157 [info] - Number of frames for camera group optimization :: 497 iter cost cost_change |gradient| |step| tr_ratio tr_radius ls_iter iter_time total_time 0 2.410110e+04 0.00e+00 5.71e+05 0.00e+00 0.00e+00 1.00e+04 0 5.98e-02 7.34e-02 1 1.364600e+03 2.27e+04 1.27e+05 5.11e-01 1.54e+00 3.00e+04 1 8.06e-02 1.54e-01 2 1.113635e+03 2.51e+02 5.58e+03 6.99e-02 1.11e+00 9.00e+04 1 9.05e-02 2.44e-01 3 1.112761e+03 8.74e-01 1.76e+02 7.21e-03 1.01e+00 2.70e+05 1 8.95e-02 3.34e-01 0020706 | 2023-11-28, 19:26:54.267438 [info] - Camera 0 :: [1, 0, 0, 0; 0, 1, 0, 0; 0, 0, 1, 0; 0, 0, 0, 1] 0020707 | 2023-11-28, 19:26:54.294396 [info] - Number of frames for camera group optimization :: 497 iter cost cost_change |gradient| |step| tr_ratio tr_radius ls_iter iter_time total_time 0 1.112761e+03 0.00e+00 9.01e+04 0.00e+00 0.00e+00 1.00e+04 0 8.33e-02 9.98e-02 1 1.038180e+03 7.46e+01 2.20e+04 3.56e+00 9.97e-01 3.00e+04 1 1.12e-01 2.12e-01 2 9.930692e+02 4.51e+01 3.60e+04 4.12e+00 9.98e-01 9.00e+04 1 1.23e-01 3.34e-01 3 9.835229e+02 9.55e+00 1.30e+04 2.47e+00 9.99e-01 2.70e+05 1 1.20e-01 4.54e-01 4 9.831213e+02 4.02e-01 5.89e+02 5.68e-01 1.00e+00 8.10e+05 1 1.10e-01 5.64e-01 5 9.831187e+02 2.65e-03 2.21e+01 4.52e-02 1.00e+00 2.43e+06 1 1.23e-01 6.88e-01 0020708 | 2023-11-28, 19:26:55.051984 [info] - Camera 0 :: [1, 0, 0, 0; 0, 1, 0, 0; 0, 0, 1, 0; 0, 0, 0, 1] 0021206 | 2023-11-28, 19:26:55.082855 [info] - Final refinement done 0021207 | 2023-11-28, 19:26:55.082879 [info] - Save parameters 0021208 | 2023-11-28, 19:26:55.104317 [info] - mean reprojection error :: 1.15312 0021209 | 2023-11-28, 19:26:55.235415 [info] - Calibration took 5009 seconds

It works.

vswdigitall commented 7 months ago

Boards were generated with your tool, here is same boards setup and calibrated successful 6 camera rig.

It was that dataset. All works. But new cameras images does not.

vswdigitall commented 7 months ago

I think boards is ok. It is fisheye issue. New cameras have very large angle of view ~170 deg.

rameau-fr commented 7 months ago

That was a good move to test on the previous dataset, it is still quite a mystery to me that the boards looks different from the one I generated but I might be wrong. Just in case could you please share the previous configuration too?

The other possibility is indeed that the fisheye calibration of OpenCV crashes on this dataset for some reasons. I will try to have a closer look tomorrow if I have time. Maybe it would worth trying on some subsamples of this new dataset or to try acquiring a new one just to see if the error is systematic. If you want you can also share with me the cam002 or cam003 to see if every single camera is affected by it.

vswdigitall commented 7 months ago

Old tool generates new kind (shifted) charuco's pattern now. I can't reproduce old kind of pattern.

rameau-fr commented 7 months ago

I have checked the very first calibration pattern 6x6 we shared with the toolbox from the beginning link and they are still the same now. So I suspect that you were maybe using a specific configuration. So maybe it is worth also trying to print some new patterns and perform some additional tests on that. Also, please try to share the configuration file from which you ran your test on the old dataset if possible. Again the problem might be caused by something else too, so do not hesitate to share anything here, you have our full support!

vswdigitall commented 7 months ago

Old dataset config:

%YAML:1.0

######################################## Print Boards Parameters ################################################### number_x_square: 6 #number of squares in the X direction number_y_square: 6 #number of squares the Y direction resolution_x: 2200 # horizontal resolution in pixel resolution_y: 2200 # vertical resolution in pixel length_square: 0.096 # parameters on the marker (can be kept as it is) length_marker: 0.072 # parameters on the marker (can be kept as it is) number_board: 6 # number of boards used for calibration (for overlapping camera 1 is enough ...) boards_index: [] square_size: 0.128 # size of each square of the board in cm/mm/whatever you want

############# Boards Parameters for different board size (leave empty if all boards have the same size) ################# number_x_square_per_board: [] number_y_square_per_board: [] square_size_per_board: []

######################################## Camera Parameters ################################################### distortion_model: 1 #0:Brown (perspective) // 1: Kannala (fisheye) distortion_per_camera : [] #specify the model per camera,

leave "distortion_per_camera" empty [] if they all follow the same model (make sure that the vector is as long as cameras nb)

number_camera: 6 # number of cameras in the rig to calibrate refine_corner: 1 # activate or deactivate the corner refinement min_perc_pts: 0.7 # min percentage of points visible to assume a good detection

cam_params_path: "None" # "../../Images_Plan/calibrated_cameras_data.yml" # file with cameras intrinsics to initialize the intrinsic, write "None" if no initialization available fix_intrinsic: 0 #if 1 then the intrinsic parameters will not be estimated nor refined (initial value needed)

######################################## Images Parameters ################################################### root_path: "/home/vsw/calib1/" cam_prefix: "cam"

######################################## Optimization Parameters ################################################### ransac_threshold: 100 #RANSAC threshold in pixel (keep it high just to remove strong outliers) ======50 number_iterations: 1000 #Max number of iterations for the non linear refinement

######################################## Hand-eye method ############################################# he_approach: 0 #0: bootstrapped he technique, 1: traditional he =====1

######################################## Output Parameters ################################################### save_path: "/home/vsw/calib1/" save_detection: 0 save_reprojection: 0 camera_params_file_name: ""

vswdigitall commented 7 months ago

News. Have tested next cam002 and it works )

e+02 -4.89e+01 7.42e+02 3.36e-01 -5.46e+00 3.85e+04 1 1.77e-02 2.60e+00 46 7.434665e+02 -4.66e+01 7.42e+02 3.31e-01 -5.20e+00 9.62e+03 1 3.61e-02 2.64e+00 47 7.315636e+02 -3.47e+01 7.42e+02 3.04e-01 -3.91e+00 1.20e+03 1 2.17e-02 2.66e+00 48 6.928788e+02 3.95e+00 4.35e+03 1.77e-01 5.60e-01 1.20e+03 1 8.77e-02 2.75e+00 49 6.790338e+02 1.38e+01 6.41e+03 1.93e-01 5.86e-01 1.21e+03 1 7.72e-02 2.83e+00 50 6.524435e+02 2.66e+01 4.26e+02 2.57e-02 1.57e+00 3.63e+03 1 6.53e-02 2.89e+00 51 6.523783e+02 6.52e-02 3.46e+00 1.37e-03 1.00e+00 1.09e+04 1 6.54e-02 2.96e+00 0022191 | 2023-11-28, 20:30:20.171361 [info] - Camera 0 :: [1, 0, 0, 0; 0, 1, 0, 0; 0, 0, 1, 0; 0, 0, 0, 1] 0022192 | 2023-11-28, 20:30:20.183560 [info] - Number of frames for camera group optimization :: 610 iter cost cost_change |gradient| |step| tr_ratio tr_radius ls_iter iter_time total_time 0 6.523783e+02 0.00e+00 2.03e+05 0.00e+00 0.00e+00 1.00e+04 0 6.92e-02 8.27e-02 1 6.429403e+02 9.44e+00 4.21e+03 7.86e-01 1.01e+00 3.00e+04 1 1.06e-01 1.89e-01 2 6.418333e+02 1.11e+00 3.56e+03 5.37e-01 1.00e+00 9.00e+04 1 1.17e-01 3.06e-01 3 6.417165e+02 1.17e-01 8.09e+02 3.14e-01 1.01e+00 2.70e+05 1 9.36e-02 3.99e-01 4 6.416856e+02 3.08e-02 4.23e+02 1.36e-01 1.01e+00 8.10e+05 1 9.19e-02 4.91e-01 5 6.416521e+02 3.35e-02 6.28e+02 1.20e-01 1.00e+00 2.43e+06 1 9.18e-02 5.83e-01 6 6.416365e+02 1.56e-02 5.63e+02 1.09e-01 1.01e+00 7.29e+06 1 1.04e-01 6.87e-01 7 6.416346e+02 1.86e-03 1.71e+02 4.39e-02 1.04e+00 2.19e+07 1 9.14e-02 7.79e-01 0022193 | 2023-11-28, 20:30:21.001512 [info] - Camera 0 :: [1, 0, 0, 0; 0, 1, 0, 0; 0, 0, 1, 0; 0, 0, 0, 1] 0022804 | 2023-11-28, 20:30:21.018196 [info] - Final refinement done 0022805 | 2023-11-28, 20:30:21.018218 [info] - Save parameters 0022806 | 2023-11-28, 20:30:21.090020 [info] - mean reprojection error :: 9.62334 0022807 | 2023-11-28, 20:30:21.180465 [info] - Calibration took 1625 seconds

Will try other 3 and 4 too and reply.

vswdigitall commented 7 months ago

So only cam002 calibrated with poor MRE. I think it is resolution and/or board alignment issue. https://cam360.tv/etc/cam002.zip Before we use 12mp cameras, now it is 5mp with larger angle.

As for board's pattern issue, it is invariant.

Will re-capture new dataset with other boards positions. I think to move closer boards to camera.

Can you advice some ideas for boards alignment for this case?

rameau-fr commented 7 months ago

Thanks for the clarifications, the fact that you have a small resolution and higher fields of view will make the pattern look significantly smaller so I would like to recommend a few thing:

use pattern with fewer corners like 5x5 or 4x4 version
try to take closer shots of the pattern to make them cover larger area of the images

I guess that it should be enough to solve your problem! Please let me know how it goes ~

rameau-fr commented 7 months ago

As another recommendation, you can also print larger boards (but making sure they are perfectly flat can be tricky). And if you still face problems, you might want to consider 1) spreading your calibration boards more to have better distributions of the points 2) pre-calibrating the intrinsic of each camera beforehand.

vswdigitall commented 7 months ago

Thank you for advises. Will answer soon.

vswdigitall commented 7 months ago

Tried to capture closer. Failed. Will build 5x5 boards.

rameau-fr commented 7 months ago

Thank you very much for the feedback. Indeed, bigger markers might help! I hope it will work out; yet another comment: I have noticed a discrepancy in the length_square and length_marker; these parameters should stay the same as for the board generation, as it might cause some issues for the detection. Only marker_size has to be adjusted

vswdigitall commented 7 months ago

length_square and length_marker fixed. Thanks.

I found decision. Have changed ransac_threshold to 10 and 3 cams calibrated. Changed to 5 then all cameras calibrated.

So it is threshold issue for 5mp sensors. For 12mp we have 100.

0127986 | 2023-12-01, 14:49:32.487707 [info] - mean reprojection error :: 6.87384

Please comment this.

New 5x5 boards at work too.

vswdigitall commented 7 months ago

And all images have small number keypoints. What kind of this issue?

000020

rameau-fr commented 7 months ago

Indeed, the detection rate seems quite low, or maybe it is because many points have been rejected by a high RANSAC threshold. I believe that you might increase the calibration's quality by using 5x5 boards or by acquiring different calibration sequences. It seems your problem is mostly solved!

vswdigitall commented 7 months ago

Yes, thanks. With this calibration i've got nice result. Hope will be better with new 9 boards. pano

vswdigitall commented 7 months ago

Hi, made new 9 boards and tests.

000194

This issue error not gone. So it is definitely not pattern issue.

But there is bad detections with small keypoints number detected and finally with huge MRE.

Now i have best result with ransac_threshold: 1 and min_perc_pts: 0.3.

Here config and output: https://cam360.tv/etc/config4.zip

Please help to improve detections.

vswdigitall commented 7 months ago

For single camera got MRE 0.7 May be i need more overlap between cameras?

Here is same moment on all cameras: ovelap4

rameau-fr commented 7 months ago

Thank you very much for all your feedback. I will try to address each question individually.

min_perc_pts is a threshold to consider a board or not. Essentially, if this variable is set to 0.5, only the boards with at least 50% of detected corners will be considered. If it is less, they will just be ignored. For robustness, given the boards you are using, you can keep this value around 0.5. Generally, you want to use a smaller threshold when you use boards with many corners and with long focal cameras.
overlap problem: You are right; perhaps the overlap between cameras is not sufficient. Do you think it would be possible to place the "columns" of boards differently? For instance, one in front, one on the left, and one on the right of the omnidirectional rig. This way, you could take better advantage of the non-overlapping calibration features of the toolbox. But make sure to have complex motions (not pure rotation or translation along a single axis).
detection issue: I believe this might be the biggest issue here... It seems Charuco is not great for detecting boards in fisheye images with large distortions. Looking at your images, I am pretty sure the boards on the side of the image are never well detected. Something that can be tested is to adjust the detector parameters: line Here is the list of parameters you can modify: Link But it may be tricky. I guess max/minMarkerPerimeterRate might make a difference, but I am not familiar with it. I would also recommend you to increase the Ransac parameters to avoid removing too many points that could be useful.
intrinsic calibration working: It can be a bit counter-intuitive, but even if the MRE is low, it does not mean that the cameras are well calibrated. For instance, it is likely that only the points located near the image center are considered, and the resulting calibration parameters are just an "overfitting" for this part of the image. So we cannot be conclusive on that.
future of MC-Calib: We are very grateful for all your feedback and would like to consider all of it in the future upgrade of our tool. We have been discussing this point with other contributors and we are now planning to incorporate AprilTag into our project, which seems much more effective for fisheye cameras. But this new version will be out at best in February or March 2024.

vswdigitall commented 7 months ago

Calibrated.

Params: min_perc_pts: 0.5, ransac_threshold: 1

I calculate intrinsic every camera individually and save all matrices to single file.

MRE for every camera is ~8 and camera_matrix and distortion very similar.

Then run 4 camera calibration with: cam_params_path: "/home/vsw/calib4/calibrated_cameras_data.yml"

0116685 | 2023-12-11, 07:23:20.559132 [info] - Camera 0 :: [1, 0, 0, 0; 0, 1, 0, 0; 0, 0, 1, 0; 0, 0, 0, 1] 0116686 | 2023-12-11, 07:23:20.559202 [info] - Camera 1 :: [0.8444522242813632, 0.4191885744071131, -0.3334387199963295, -0.004591461698019311; -0.3815622909895132, 0.03389063902843081, -0.9237216261845768, -0.06048093209337892; -0.3759131003318353, 0.907266423754774, 0.1885655783172813, -0.01179437767127926; 0, 0, 0, 1] 0116687 | 2023-12-11, 07:23:20.559224 [info] - Camera 2 :: [0.6940789095245901, 0.01507375980869724, -0.7197410986725615, -0.03376616181608776; 0.002131805557625711, -0.9998194137716473, -0.01888372977946927, -0.04778605522943915; -0.7198957721491498, 0.01157245049888925, -0.6939857027570742, -0.07193595121956944; 0, 0, 0, 1] 0116688 | 2023-12-11, 07:23:20.559247 [info] - Camera 3 :: [0.8469469213441342, -0.3962314961227271, -0.3545158866765191, -0.02389288741582184; 0.3838721250554823, -0.005627282485804, 0.923369116495247, 0.01338535465674894; -0.3678628875424664, -0.9181333972643465, 0.1473362168536589, -0.05750683408059914; 0, 0, 0, 1] 0119976 | 2023-12-11, 07:23:20.619255 [info] - Final refinement done 0119977 | 2023-12-11, 07:23:20.619280 [info] - Save parameters 0119978 | 2023-12-11, 07:23:20.744020 [info] - mean reprojection error :: 6.54986 0119979 | 2023-12-11, 07:23:29.450697 [info] - Calibration took 8134 seconds

Thank you for advises and your excellent software.

rameau-fr / MC-Calib

Calibrate error objectPoints.total() == imagePoints.total() #51

System information (version)

Vision system

Describe the issue / bug

leave "distortion_per_camera" empty [] if they all follow the same model (make sure that the vector is as long as cameras nb)

fix_intrinsic: 0 #if 1 then the intrinsic parameters will not be estimated nor refined (initial value needed)

leave "distortion_per_camera" empty [] if they all follow the same model (make sure that the vector is as long as cameras nb)