Inaccurate Calibration

[KenR22]

Hi,

I have been trying to run the calibration on a 7x4 checkerboard. However, there seem to be huge discrepancies between cameras. When I reproject some of the key points using the annotation tool, the error goes very high(>30). I was wondering if you could take a look at the calibration and give some suggestions.

https://drive.google.com/file/d/1DgYuZCVzb2UBp2KxiFrPaGtPA5caMG-P/view?usp=sharing

Thanks, Ken

[JARVIS-MoCap]

Hi Ken,

Thanks for sharing those videos! First of all the video of Cam3 is 41 frames longer than the ones of all the other cameras, so something has definitely gone out of sync a little bit there. Apart from that it seems that some cameras are calibrated relatively accurately, while some others are off quite a bit. Here are some troubleshooting steps/improvements that I think might help make it more consistent:

1. I'm not sure if this even is an issue in the recordings you shared, but make sure your checkerboard is perfectly flat (maybe by glueing it to the clipboard.
2. If possible it is probably a good idea to switch to a bigger checkerboard. It becomes super pixelated then far away from the cameras, I'm sure that is hurting the accuracy quite a bit.
3. Switch to a two step calibration procedure. Meaning you first record one video per camera for intrinsics calibration and then do the pairwise calibration in the way you're already doing it. That way you can get way closer to the cameras with the checkerboard and get a better estimate of the lenses distortion etc.
4. Wiggle the checkerboard more during your calibration recordings. Currently the poses of the checkerboard are all very similar, by moving the board around more you can get a lot more accurate and robust calibrations.
5. I already mentioned it abvove, but make sure that your cameras are synchronized correctly. It seams like that's the case for all of them except Cam3, but double checking is probably a good idea.

I hope this is a good starting point, but please let me know if you have any further questions!

Best, Timo

[KenR22]

This is great!

I have tried all that and getting error like below

Do you think this is an acceptable range?

The projection tool seems still a lot off when annotating.

Thanks

[JARVIS-MoCap]

The reprojection-errors you're getting seem to be in a very acceptable range. I would expect the projections to look pretty decent judging by those numbers, so I'm not really sure what's going wrong. If you want you can definitely share the new calibration videos with me and I can have a look again.

I noticed one more thing with your old calibration attempt though. It seems that you are calibrating Cam8 with respect to Cam10 instead of Cam9. It's important to select the same primary camera for every pair that you are calibrating (Cam9 in your case). I can see how this is not super intuitive in the GUI right now, but that's definitely something you should fix if you carried that over to your new calibration attempts.

Playing around with the calibration you uploaded some more I also get pretty decent projections (with the exception of Cam8 of course). I tested it by annotating the calibration recordings though, so could it be that there is an issue (synchronization, cameras have moved etc.) with the real recordings you are trying to annotate? One easy way to test if synchronization is the issue is to annotate a static point in the setup instead of the moving subject. If that improves the projection quality it strongly points to sync issues.

Hope this helps, Timo

[KenR22]

These are very helpful. I will check for the points you mentioned.

Thanks, Ken

[KenR22]

I think the issue was with one particular camera and we have decided to move forward without that camera. When we exclude that camera the error is in acceptable range.