3D reconstruction right on distance but wrong on degree

[KJ-Chiu]

Hi, I follow the steps and got rmse value around 1. However when I rebuilt my chessboard to 3D figure, it became parallelogram not rectangle. I had a 5 x 4 chessboard with 90mm world scale, the rebuilt figure got 455mm and 356mm borders which are pretty close. But the angles are 35 and 55, they should be 90.

Is there anything I should do before transforming the triangulation xyz points to 3D figure?

[TemugeB]

RMSE of around 1 might be too high. Try calibrating again a bit more carefully. Maybe try to move the calibration pattern closer. By default, this package provides R, T with camera0 as the origin. From here, you can manually set any new origin by simply writing down the transformation matrices by hand.

Another problem that often occurs is that you are using R,T in the reverse order. From one coordinate to another, you would usually transform like this:

p = R @ p' + T

But in reverse, this would be:

p' = R.transpose @ p - R.transpose @ T

So make sure you got the right R, T. In other words, is your calibration pattern points p or p'.

[KJ-Chiu]

First, I made a mistake of calculating degree, the degree turned to 93 after the fix. Second, I realized my images for stereoCalibrate was not perfect for calibrateCamera, it was too small for camera calibrate (image below).

Therefore I calibrated cameras separated and made chessboard almost filled up the screen to get more precision matrix and distortion. I got 0.3 RMSE for the first camera, 0.7 for the second camera and 0.8 for the stereo calibrate.

The end up result is well enough for me: 449mm x 358mm with 89.9 degree

Thanks for the advice! You made me turn back to check my calibrate workflow again.