Strange calibration results and reprojection

[TuShuqi]

Thank you very much for your hard work, and it is very meaningful in sensor calibration! But when I tried this project, I encountered some problems and needed your help!

Due to the lack of actual LiDAR and calibration office, I attempted to calibrate it in Gazebo simulation. The LiDAR used is Velodyne HDL-32E. The camera parameters used are as follows:

Height: 512
Width: 640
Disruption_model: "plumb'bob"
D: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
K: [554.254691191187, 0.0, 320.5, 0.0, 554.254691191187, 256.5, 0.0, 0.0, 1.0]
R: [1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0]
P: [554.254691191187, 0.0, 320.5, -38.79782838331, 0.0, 554.254691191187, 256.5, 0.0, 0.0, 0.0, 1.0, 0.0]
Binning_x: 0
Binning_y: 0
ROI:
X_offset: 0
Yyoffset: 0
Height: 0
Width: 0
Do-rectify: False

When I follow the readme tutorial for calibration, I can correctly capture the calibration board in the camera and the calibration board in the LiDAR. As shown in the following figure:

I collected a total of eight samples and the final results are as follows:

---- Calculating average reprojection error on 8 samples ---- 
   1/  8 | dist= 0.000m, dimerr=  70.300mm | error:   0.258pix  -->   3.262mm
   2/  8 | dist= 0.000m, dimerr=  61.920mm | error:   0.040pix  -->   0.499mm
   3/  8 | dist= 0.000m, dimerr=  58.280mm | error:   0.208pix  -->   2.156mm
   4/  8 | dist= 0.000m, dimerr=  88.620mm | error:   1.001pix  -->  14.972mm
   5/  8 | dist= 0.000m, dimerr=  69.050mm | error:   0.221pix  -->   2.957mm
   6/  8 | dist= 0.000m, dimerr=  77.000mm | error:   0.192pix  -->   2.141mm
   7/  8 | dist= 0.000m, dimerr=  77.650mm | error:   0.205pix  -->   2.371mm
   8/  8 | dist= 0.000m, dimerr=  74.440mm | error:   0.291pix  -->   3.116mm

Although the calibration result does not seem to match the actual pose relationship of my sensor, I still tried the final reprojection and the final result is as follows:

---- Calculating average reprojection error on 8 samples ---- 
   1/  8 | dist= 0.000m, dimerr=  70.300mm | error:   0.258pix  -->   3.262mm
   2/  8 | dist= 0.000m, dimerr=  61.920mm | error:   0.040pix  -->   0.499mm
   3/  8 | dist= 0.000m, dimerr=  58.280mm | error:   0.208pix  -->   2.156mm
   4/  8 | dist= 0.000m, dimerr=  88.620mm | error:   1.001pix  -->  14.972mm
   5/  8 | dist= 0.000m, dimerr=  69.050mm | error:   0.221pix  -->   2.957mm
   6/  8 | dist= 0.000m, dimerr=  77.000mm | error:   0.192pix  -->   2.141mm
   7/  8 | dist= 0.000m, dimerr=  77.650mm | error:   0.205pix  -->   2.371mm
   8/  8 | dist= 0.000m, dimerr=  74.440mm | error:   0.291pix  -->   3.116mm

Calibration params (roll,pitch,yaw,x,y,z): -1.5710,0.0008,-1.5709,0.2440,-0.0096,-0.1140

Mean reprojection error across  8 samples
- Error (pix) =  0.302 pix, stdev =  0.292
- Error (mm)  =  3.934 mm , stdev =  4.544

[ INFO] [1714097377.784034410]: Projecting points onto image for pose #3

This looks very strange. I have tried many times, and each time I get the same extrinsic parameters, and the reprojection is also similar. This result is very strange and bothers me a lot. I couldn't find what problem led to this result. I wonder if you have encountered a similar situation? Can you provide me with some help?

[chinitaberrio]

The result is common with you have the wrong camera intrinsic parameters or camera model.

[LiUziYu07]

thank you for pointing out this problem. WE also encounter this problem. May I know if you have fixed this problem? I have checked that the intrinsic parameters are correct. What does camera model mean? For our project, we use the norm pinhole camera. Kindly please suggest.

[chinitaberrio]

What's the FOV of your camera?

[LiUziYu07]

The FOV of my camera is about 120 degrees. Thank you

[LiUziYu07]

Thank you for your help. We follow the instruction of this code. and have collected 23 samples with board dim err about 40mm. However, once we run the final estimated calibration. As shown in picture, no lidar point and we found that the calculation dist is always equal to zero.

[chinitaberrio]

We calibrate 120 FOV cameras as fisheye. Can you project the point cloud into the image by using the values from the calibration package to inspect if the numbers make sense?

[LiUziYu07]

Our camera is normal pinhole camera,but the FOV is 120degrees. Shall we change the model to fisheye although it is pinhole one. Thank you.

Today, we have collected 20 samples whose dim err within 45 but the results shown today is as figure Our camera is normal pinhole camera,but the FOV is 120degrees. Shall we change the model to fisheye although it is pinhole one. Thank you.

Today, we have collected 20 samples whose dim err within 45 but the results shown today is as figure