Camera improvements - Githubissues

vlachoudis / bCNC

GRBL CNC command sender, autoleveler and g-code editor

GNU General Public License v2.0

1.54k stars 528 forks source link

Camera improvements #1882

Open jasonharper opened 2 months ago

jasonharper commented 2 months ago

'aligncam' and 'webcam' config settings can now be a string instead of an integer. This is needed to access video sources that OpenCV cannot auto-detect, and therefore does not assign an index number for. (My personal use-case is a WiFi camera, intended for use on a drone, that is accessed via a rtmp:// URL. It might also be useful as a stable identifier for a particular camera, that remains valid even if other cameras are connected or disconnected, but I don't know a general way to determine the string for an indexed camera.)
Added 'PREFIX_threaded' config option for each camera: if 1, the camera is read continuously in a background thread, and the most recent image is used when one is requested. This is needed for cameras that return the oldest available frame, rather than the newest, when read less frequently than their natural FPS rate. (Without this option, my WiFi camera has about 15 seconds of lag, making it utterly unusable.)

Harvie commented 2 months ago

Hello, thanks for improvements. I think it mostly makes sense, but i am not sure about the threading thing... doesn't OpenCV have some API to drop all the buffered frames and only get the latest one?

Maybe we can just read all the frames in buffer each time, that seems as a most straightforward solution to me:

ret = True
while ret:
    ret, frame = cam.read()

What about disabling the buffer completely? eg.:

cap.set(cv2.CAP_PROP_BUFFERSIZE, 0)

does that help in your case?

Mabye there is some further reading: https://stackoverflow.com/questions/43665208/how-to-get-the-latest-frame-from-capture-device-camera-in-opencv

This page https://stackoverflow.com/questions/59924971/is-there-a-way-to-show-last-frame-in-opencv-python

also proposes way to retrieve index of latest frame in buffer and retreive directly frame at that index...

jasonharper commented 2 months ago

CAP_PROP_BUFFERSIZE reportedly doesn't work on all cameras - and I'm pretty sure that cameras like mine are in the non-working category: the problematic buffering is happening in the camera, where OpenCV has no control over it.

Your second link is talking about getting the last frame of a video file, it is not applicable to cameras.

If your concern is that the background thread takes too much CPU time, please note that the laptop I'm testing this on shows around 5% CPU utilization with the camera active in bCNC.

Harvie commented 2 months ago

and I'm pretty sure that cameras like mine are in the non-working category

Can you try please?

jasonharper commented 2 months ago

I finally got a chance to try CAP_PROP_BUFFERSIZE as you suggested. It did actually help - but it only got the lag down to 4 seconds, which is still unusable.