[HOW-TO] Improve video quality of Raspberry Pi Camera v3 for detecting aruco markers in very high or low light conditions

[buscainod]

I use the Raspberry Pi Camera v3 to capture the area beneath a drone, to detect an aruco marker and land precisely. The drone's vibrations cause wave-like distortions in the video. To mitigate this problem I configure the camera as follows:

self.camera.create_preview_configuration(main={"format": 'RGB888',
                           "size": (640, 480)},
                           raw={"size": (2304, 1296)}))

self.camera.set_controls({"FrameDurationLimits": (45000, 45000),
                          "Saturation": 0, "ExposureTime": 200,
                          "AfMode": controls.AfModeEnum.Manual, "LensPosition": 0})

This configuration helps a lot in recognizing the marker but, in very sunny conditions, the marker appears too bright, making it hard to detect.

I tried changing the exposure time during the recognition process but that doesn't solve the issue in some situations. How can I further improve the quality to ensure reliable detection of the marker in bright light and low-light conditions?

Here an example:

[davidplowman]

The v3 camera has a built-in HDR mode which you could try. I don't know if it would help. Check out section 9.6 of the manual.

In your comments above, I see you are changing the exposure time but not the analogue gain. Presumably the analogue gain is increasing because of the low exposure time? You might want to try setting both. The minimum analogue gain value for the v3 camera is 1.12x, IIRC.

Finally, the AE algorithm has a "highlight" metering mode which aims to avoid over-exposing highlights in an image by lowering the overall exposure. You could try that with something like

self.camera.set_controls({'AeConstraintMode': controls.AeConstraintModeEnum.Highlight})

(and you can switch between that and the Normal mode at runtime).

[buscainod]

Sorry for the late reply, but I would like to share with you the final results I got.

I tried to create an algorithm to test different combinations of analog gain, exposure values, and AE algorithm, then select the best combination based on the brightness level and how often the ArUco marker was detected. Unfortunately, the results were not entirely positive because the algorithm occasionally failed to select the correct combination.

During these tests, I had the chance to try the Raspberry Pi Camera v2, which surprisingly did not require any special configuration to recognize the marker. In fact, the frames did not present wave-like distortions, allowing the marker to be detected even with high exposure values.

I now believe that the problem I encountered was related to the autofocus capability of the v3 camera. Although I had set the focus to infinity, the motorized autofocus mechanism seemed to be affected by the vibrations of the drone, causing distortions in the frames. When I switched to the v2 camera, which does not have autofocus, the problem was resolved!

Thank you very much for your help!