search

mfxox / ILCC

Intensity-based_Lidar_Camera_Calibration
BSD 2-Clause "Simplified" License
432 stars 122 forks source link

Detected Corners from point clouds and monocular camera can not match #3

Closed Rockyzyk closed 6 years ago

Rockyzyk commented 6 years ago

I use VLP16 and Flea3 monocular camera to do the calibration. Corners detection of camera and VLP16 respectively is finish without any problems. But after run the commend: _from ILCC import LM_opt LM_opt.cal_extparas()

the calibration result is output as following: 0020_cal_backproj

I tried to change increase _jdc_threratio and _agglomerative_cluster_thratio in config.yaml, but it is no works.

Could you please check the problem, and answer it? Thanks a lot! If you need any file or more information please ask me.

mfxox commented 6 years ago

It's highly possible that 3D corners may not be correctly detected due to a wrong detection of chessboard' point cloud.

Can you check whether 3D corners were correctly detected with the following code?

    from ILCC import utility
    utility.vis_ested_pcd_corners(ind=1)

And how many image-pcd pairs did you use?

Rockyzyk commented 6 years ago

Thanks for reply and answer! Here is my figure after running the code: from ILCC import utility utility.vis_ested_pcd_corners(ind=1) image 1 I think it can detect chessboard correctly.

My image-pcd pairs is 7 pairs, but the chessboard doesn't move, which is in same relative position between camera and VLP16 in these 7 pairs. it is OK? And could you please explain more about _jdc_threratio and _agglomerative_cluster_thratio in config.yaml. What is their scope? And how should I adjust them in this situation?

mfxox commented 6 years ago

From the image you uploaded, both the point cloud of chessboard and 3D corners are correctly detected as you thought. I just noticed that the counting order of corners on the image differs from the request for running the program. This will cause the mis-correspondence between image-pcd corners and further cause the failure of the optimization. Currently, the counting order of 2D and 3D corners obeys to the rule: 1. starts from the lower left; 2. counts from lower left to upper right with the zigzag order. You can check a sample image shown here. Can you try to reorder the image corners and run the optimization again?

I am not sure why you didn't move the chessboard in 7 pairs. If you don' t change the relative position/pose of the chessboard to increase variation, what is the meaning to increase the image-pcd pairs.

As for _jdc_threratio and _agglomerative_cluster_thratio, they relate to the segmentation of the point cloud to get the chessboard. As the chessboard seems to be segmented correctly, I think it's fine of these two parameters for your situation.

Rockyzyk commented 6 years ago

By the way, I use OpenCV the detect image corners, and the first point is not “lower left” is “lower right”, maybe that is the reason, I will try to fix it and reply you. Furthermore, the setting of corner size for OpenCV is different with MATLAB. For example, your checkboard size is (8,6) for MATLAB but is should be changed to (7,5) in OpenCV. I found the code doesn't deal with it. If you have time, you can fix it!

Rockyzyk commented 6 years ago

I just received you reply! Thanks a lot! Perhaps, the first detection point is mean reason as we though. I will check now!

Rockyzyk commented 6 years ago

I changed the first detection point to “lower left” in the image, and it works! Thanks so much for help!

nonlinear1 commented 3 years ago

@Rockyzyk I found that your checkerboard is different from the one which is given by mfxox. I think the program must changed according to your new partern checkerboard. And Could you tell me how can change your program? I was faced the same problem and I want change a different checkerboard different from the one given by mfxox. Thank you very much!