stereo calibration underperforming wrt mono calibration

[lsorgi]

hello I have a camera system consisting of 2 synchronized sensors with diverging optical axis (fov overlap ~30%)

if I run the intrinsic calibration of both cameras independently the result are very nice...low reprojection error, low count of outliers if I run the calibration of both cameras simultaneously then the results becomes more fishy... high reprojection error, high count of outliers

here are the reports from the 3 runs stereo-report.pdf cam0-report.pdf cam1-report.pdf

any hint? what could be the reason for such deterioration of calibration metrics?

[goldbattle]

You need to have the cameras use the same timestep and take images at near the same time (re your issue https://github.com/ethz-asl/kalibr/issues/633). The board needs to be in the same place in 3d space when both cameras collect an image. If you are unable to sync the camera timesteps and trigger then relatively near each other, then you could manually trigger and keep the board fixed at each timestep you manually trigger. Then can use bagcreater to get your rosbag for processing.

You can consider the case that you have two images and the board has moved between them. This is the same as having a non-rigid transformation between the stereo pair!

[lsorgi]

The cameras are synchronized, captured using the multicamera raspberry pi shield. Images are names as specified in the wiki TIMESTAMPNANOSEC.png timestamp is the same for both camera streams

any other issue which might lead to stereo calibration failure?

[lsorgi]

fixed. error on our side. camera synchronization was broken