Error: Seems that the camera light image is the same as the camera grid image, try a longer exposure time and recapture

[EnochYing]

Hi,

When I use the software built from source to capture the images, I always get the error "Seems that the camera light image is the same as the camera grid image, try a longer exposure time and recapture". But the two images (light image and grid image) are successfully displayed in the two relevant boxes on the interface. I tried the longest exposure time but this does not help. It is strange that the windows executable application works pretty good for the same image capturing environment (same world environment and same camera and projector configurations). What are the possible causes and how can I fix this? Thanks very much.

Enoch

[BingyaoHuang]

@ChanningYing I had the same problem when I upgraded MATLAB 2019b to 2021a. But I fixed this problem in Release v0.2, and it worked well in my setup. To find out the causes of this issue you may compare the difference between the current source and Release v0.2 source. Please let me know if there's anything I can fix after that.

[EnochYing]

@BingyaoHuang Thanks very much for the reply. Yes, the release v0.2 application does not have such issue. I am using the source code from the master branch. I will test the Release v0.2 source. Actually I solved this issue by commenting relevant code that checks the difference between captured light image and grid image.

Another problem is that when I used a camera with a high resolution (1920 X 1080), I cannot get good calibration results. I tried your suggestion about adjusting the kernel sizes (2, 3, 4, ... , 12) in segColorGrid and bw2skele. But unfortunately, the results were still bad. I analyzed the processing results of each image set and found that some results are pretty good but for some others the errors are very big. In summary, the processing seems not very robust. Do you have any clue to fix this?

[BingyaoHuang]

@ChanningYing

Actually I solved this issue by commenting relevant code that checks the difference between captured light image and grid image.

If the captured light image and grid image are the same, the calibration may fail. Commenting the code may suppress the error message but how about the the light image and grid image issue, could you double check this?

I analyzed the processing results of each image set and found that some results are pretty good but for some others the errors are very big

I can take a look, could you post some intermediate results?

[EnochYing]

@BingyaoHuang

1. The master source and Release v0.2 can produce very different calibration results (see the two figures below). Which version should I use?

Results from master source

Results from Release v0.2

2. After collecting a set of images (e.g., 20 image pairs), I have to try many different subsets of images to determine the best subset that can produce small or acceptable reprojection errors. I just cannot use the full set of images because the reprojection errors can be extremely big (see Figures above). I think it may be because there exist some low-quality images to the algorithm? Do you have any suggestions about picking up high-quality images? My current solution is running the entire set of images, analyzing the reprojection errors of each pair of images, and then excluding all the image pairs that produce big errors. Repeating the above process until acceptable errors are reached. But I am not sure this is "scientific" because when I use a small set of images that consists of the "low-quality" images only, the reprojection errors can also be small (much smaller than their results in the full-set image experiment).

3. I am now using a projector-camera system to project construction layouts on construction site. I calibrated the system using some different subset of images that produce small reprojection errors. But it is strange that smaller reprojection error does not mean more accurate projection of the construction layouts (see the table below). All the trials have the exact same experiment environment except the used calibration parameters, which means the projection differences come from the calibration process only. What do you think are the potential reasons? I further analyzed the differences of interior parameters of the camera and projector obtained from Trials 1, 2, and 5, which have very close reprojection errors (see the figure below). The differences for some parameters look remarkable.

[BingyaoHuang]

@ChanningYing Thank you for sharing the results.

Which version to use, master source or Release v0.2?

Use the one that gives the lowest reprojection errors.

Do you have any suggestions about picking up high-quality images?

I used the same strategy as yours, by excluding all the image pairs that produce big errors MATLAB's Camera Calibrator App also does this). In the future, we can use RANSAC to automatically pick the best subsets.

Smaller reprojection error does not mean more accurate projection of the construction layouts.

It may be the case. Make sure to put the calibration board at a distance that is similar to the construction layouts.

Also, try checking the "verbose" option in calibration tab before you click "calibrate", and see the debug information, e.g., see if the projected and captured color grids are properly matched. Sometimes bad matches may cause large calibration errors.

[EnochYing]

@BingyaoHuang Thank you very much for your prompt reply.

Use the one that gives the lowest reprojection errors.

Okay. I will use Release v0.2 version.

It may be the case. Make sure to put the calibration board at a distance that is similar to the construction layouts.

Currently I put the calibration board close to the projector, about 0.8m for the calibration. The target projection surface in my scenario is a little distant to the projector, about 4 meters. The projector's resolution is only 1280 * 800. If I put the calibration board close to the projection surface, the lines of the color grids projected on the board become relatively thick. Would this affect your algorithm's performance?

Also, try checking the "verbose" option in calibration tab before you click "calibrate", and see the debug information, e.g., see if the projected and captured color grids are properly matched. Sometimes bad matches may cause large calibration errors.

This seems a good way to identify bad image pairs. I will try.

Thanks again for your patience.

[BingyaoHuang]

@ChanningYing

If I put the calibration board close to the projection surface, the lines of the color grids projected on the board become relatively thick. Would this affect your algorithm's performance?

Yes, a sparser color grid may also contain less nodes in the camera-captured image, and further reduces #nodes for calibration. But give it a try and see what happens.

This seems a good way to identify bad image pairs. I will try.

Recently I tried the calibration app on a new setup, and had a similar large reprojection errors issue like yours. I connected my camera using USB, and installed Canon EOS Webcam Utility, so MATLAB Support Package for USB Webcams can recognize my Canon camera. When I calibrate this setup, my stereo reprojection errors were very high (~30, like yours), then I debugged and found that the frames obtained by MATLAB's webcam is upside down, leading to mismatched color grid points. Note that Canon EOS Webcam Utility worked well in other apps, e.g., Windows Camera App. So, try debugging and you may find the cause.