`cv2.VideoCapture` implementation of `RGBCamera`

[Victorlouisdg]

This OpenCV way to access cameras is through cv2.VideoCapture. This provides an easy way to use webcams, usb cameras and videos as cameras. I'm not sure whether we want this in the airo-camera-toolkit (might be a case of "feature creep"). However I wrote this once to test some code at home, so I'll just leave it here:

from __future__ import annotations

from typing import Any, Tuple
import numpy as np
from airo_camera_toolkit.interfaces import RGBCamera
from airo_camera_toolkit.utils import ImageConverter
from airo_typing import CameraIntrinsicsMatrixType, NumpyFloatImageType, OpenCVIntImageType
import cv2

class OpenCVVideoCapture(RGBCamera):
    """Wrapper around OpenCV's VideoCapure so we can test the camera interface without external cameras."""

    def __init__(
        self,
        video_capture_args: Tuple[Any] = (0,),
    ) -> None:
        self.video_capture = cv2.VideoCapture(*video_capture_args)
        if not self.video_capture.isOpened():
            raise RuntimeError("Cannot open camera")

        self.fps = self.video_capture.get(cv2.CAP_PROP_FPS)

    def __enter__(self) -> RGBCamera:
        return self

    def __exit__(self, exc_type: Any, exc_value: Any, traceback: Any) -> None:
        self.video_capture.release()

    def intrinsics_matrix(self) -> CameraIntrinsicsMatrixType:
        # TODO: Warn about unknown intrinsics?
        return np.identity(3)

    def get_rgb_image(self) -> NumpyFloatImageType:
        ret, image = self.video_capture.read()
        if not ret:
            raise RuntimeError("Can't receive frame (stream end?). Exiting ...")

        return ImageConverter.from_opencv_format(image).image_in_numpy_format

if __name__ == "__main__":
    import cv2

    camera = OpenCVVideoCapture()

    while True:
        image = camera.get_rgb_image()
        print(image.shape)
        image = ImageConverter.from_numpy_format(image).image_in_opencv_format

        cv2.imshow("VideoCapture", image)
        key = cv2.waitKey(10)
        if key == ord("q"):
            break

[tlpss]

I think we can include this? seems useful and low maintenance. Would not start to support intrinsics calibration though untill we have a clear use case, and I would throw an error in the intrinsics matrix function

[m-decoster]

I also think this would be a useful addition. It's nice to have an implementation that can work with built-in webcams or video files if you just want to do some computer vision experiments

[Victorlouisdg]

I agree we don't need to support intrinsics calibration yet, but instead, I would allow it as a constructor argument in case you know the intrinsics.