The choise of checkerboard seems to make a big difference, the default 7x7 does not seem to work

[janamach]

Hi,

I thought I should post this here until this information is lost forever.

In braid, the default checkerboard in the checkerboard calibration option is 7x7 corners:

Historically, we've been using 8x6 ones, so the default option had to be changed every time. Since I calibrate and re-calibrate on regular basis, I made a 7x7 ones just to save those two extra seconds. Unfortunately, it took me weeks to figure out that the format of the checkerboard was the problem in the next steps of VR calibration.

This is just a suggestion, perhaps for the next release: if it is not too much work, can you change the default 7x7 to 8x6 or at least write a small note recommending against using 7x7 for multi camera tracking?

[astraw]

The default is empty, AFAICS.

[astraw]

Sorry I did find the default and changed it in 74db4909fc96d53740f5bb7cace2d0e74fd8873a.

I did not know that a square checkerboard would cause problems, thanks for the report!

[astraw]

@janamach although I did change the default, I am still puzzled by this. Do you have the YAML files from ~/.config/strand-cam/camera_info/ from made with 7x7 vs 8x6 checkerboards? If this makes a difference, these files should show it.

[janamach]

The calibration made on 20210224 was performed with 7x7 checkerboard, the two other versions were made with 8x6:

strawlab@fishtrax3:~/.config/strand-cam/camera_info$ ll
total 64
drwxrwxr-x 1 strawlab strawlab 1024 Feb 26 17:45 ./
drwxrwxr-x 1 strawlab strawlab   22 Feb 24 16:33 ../
-rw-rw-r-- 1 strawlab strawlab  826 Feb 24 16:33 Basler_22005677.20210224_163307.yaml
-rw-rw-r-- 1 strawlab strawlab  827 Feb 26 10:58 Basler_22005677.20210226_105806.yaml
-rw-rw-r-- 1 strawlab strawlab  826 Feb 26 17:37 Basler_22005677.20210226_173746.yaml
-rw-rw-r-- 1 strawlab strawlab  826 Feb 26 17:37 Basler_22005677.yaml
-rw-rw-r-- 1 strawlab strawlab  825 Feb 24 16:36 Basler_22139107.20210224_163549.yaml
-rw-rw-r-- 1 strawlab strawlab  823 Feb 26 11:00 Basler_22139107.20210226_105952.yaml
-rw-rw-r-- 1 strawlab strawlab  822 Feb 26 17:40 Basler_22139107.20210226_174032.yaml
-rw-rw-r-- 1 strawlab strawlab  822 Feb 26 17:40 Basler_22139107.yaml
-rw-rw-r-- 1 strawlab strawlab  823 Feb 24 16:38 Basler_22139109.20210224_163809.yaml
-rw-rw-r-- 1 strawlab strawlab  824 Feb 26 11:02 Basler_22139109.20210226_110152.yaml
-rw-rw-r-- 1 strawlab strawlab  826 Feb 26 17:42 Basler_22139109.20210226_174217.yaml
-rw-rw-r-- 1 strawlab strawlab  826 Feb 26 17:42 Basler_22139109.yaml
-rw-rw-r-- 1 strawlab strawlab  826 Feb 24 16:41 Basler_22139110.20210224_164129.yaml
-rw-rw-r-- 1 strawlab strawlab  825 Feb 26 11:04 Basler_22139110.20210226_110410.yaml
-rw-rw-r-- 1 strawlab strawlab  824 Feb 26 17:45 Basler_22139110.20210226_174444.yaml
-rw-rw-r-- 1 strawlab strawlab  824 Feb 26 17:45 Basler_22139110.yaml

[astraw]

I looked at the files for one camera. Maybe there are other cameras with bigger differences, but for the the camera I looked at, the difference looks within the range of insignificant noise. I'm not convinced that this was a problem. Have you tried running the later stages of calibration with the 7x7 checkerboard calibrations but new data that otherwise worked well? This would tell us if it is the 7x7 checkerboard data that is indeed problematic or some other factor which now remains unknown.

Here is ~/.config/strand-cam/camera_info/Basler_22005677.20210224_163307.yaml:

---
image_width: 1280
image_height: 1024
camera_name: Basler_22005677
camera_matrix:
  rows: 3
  cols: 3
  data:
    - 1177.5570250987349
    - 0.0
    - 648.8318646696578
    - 0.0
    - 1178.6553786979135
    - 541.8772863284457
    - 0.0
    - 0.0
    - 1.0
distortion_model: plumb_bob
distortion_coefficients:
  rows: 1
  cols: 5
  data:
    - -0.3636234577763582
    - 0.15677226056844668
    - -0.0002671322417501036
    - -0.0010328350785593596
    - 0.0
rectification_matrix:
  rows: 3
  cols: 3
  data:
    - 1.0
    - 0.0
    - 0.0
    - 0.0
    - 1.0
    - 0.0
    - 0.0
    - 0.0
    - 1.0
projection_matrix:
  rows: 3
  cols: 4
  data:
    - 1177.5570250987349
    - 0.0
    - 648.8318646696578
    - 0.0
    - 0.0
    - 1178.6553786979135
    - 541.8772863284457
    - 0.0
    - 0.0
    - 0.0
    - 1.0

And here is ~/.config/strand-cam/camera_info/Basler_22005677.20210226_173746.yaml:

---
image_width: 1280
image_height: 1024
camera_name: Basler_22005677
camera_matrix:
  rows: 3
  cols: 3
  data:
    - 1185.9206727178017
    - 0.0
    - 642.0042949330018
    - 0.0
    - 1184.8275949555226
    - 534.4284544841271
    - 0.0
    - 0.0
    - 1.0
distortion_model: plumb_bob
distortion_coefficients:
  rows: 1
  cols: 5
  data:
    - -0.35369427889332386
    - 0.15167233917955548
    - 0.0016991604953059076
    - 0.00022727333179948606
    - 0.0
rectification_matrix:
  rows: 3
  cols: 3
  data:
    - 1.0
    - 0.0
    - 0.0
    - 0.0
    - 1.0
    - 0.0
    - 0.0
    - 0.0
    - 1.0
projection_matrix:
  rows: 3
  cols: 4
  data:
    - 1185.9206727178017
    - 0.0
    - 642.0042949330018
    - 0.0
    - 0.0
    - 1184.8275949555226
    - 534.4284544841271
    - 0.0
    - 0.0
    - 0.0
    - 1.0
    - 0.0

[janamach]

Have you tried running the later stages of calibration with the 7x7 checkerboard calibrations but new data that otherwise worked well?

Not sure what you mean. In the manual calibration workflow, the calibration yaml files are used almost immediately to generate the unaligned XML. I did not try manually editing the working aligned XML file to incorporate the 7x7 camera calibration data, if that's what you're asking.

With AprilTag calibration, the YAML files are also used only once when running apriltags_solve_fishbowl.py. Neither 7x7 nor 8x6 produced anything meaningful, but the generated plots looked different: compare the earlier and the latter plots I posted here. You can also take a look at the data if you like:

7x7

/home/strawlab/ros/strawlab-vr-kinetic-catkin/src/flycave/launch/fishtrax2/apriltag-cal-2021-02-24/

8x6

/home/strawlab/ros/strawlab-vr-kinetic-catkin/src/flycave/launch/fishtrax2/apriltag-cal-2021-02-26/

All in all, I agree this is interesting, but I am afraid we can keep working on the calibration problems indefinitely. While I don't think we should keep spending our time on this particular issue, perhaps it would be an interesting internship or thesis project for a student interested in computer vision? The student could use a set of different checkerboards and analyze the calibration output accuracy. This would be a very straightforward project that would be useful for us. Just a thought.

[astraw]

I agree we do not need to solve this right now (and the issue is closed, after all) but in the same spirit of you documenting the issue so it is "not lost forever", I also want to log my doubt about 7x7 checkerboards being the cause of failed calibrations. Especially because the checkerboard calibration results I checked between 7x7 and 8x6 were very similar to each other.