Vanilla OpenCV example code won't open ArduCam on RPI

[UberEclectic]

I have the SDK library installed:

/usr/lib/libArduCamLib.so /usr/lib/libArduCamLib.so.2 /usr/lib/libArduCamLib.so.2.0.0

I am trying to use a vanilla OpenCV example, which attempts to open the "default" simply camera as follows:

// Initialize camera VideoCapture cap(0); if (!cap.isOpened()) { cerr << "Couldn't open video capture device" << endl; return -1; }

The ArduCam-provided RPI cpp code does not provide any example on how to extend generic OpenCV code to leverage the SDK libraries to be able to open the camera.

Can you please provide an example?

I expect I need to link the SDK library and specify the appropriate camera yml, but I do not see how to do it plainly. :-( Yes, I see the "proprietary" way to open the camera in .../ArduCAM_USB_Camera_Shield/RaspberryPi/Cpp/Streaming_demo/ArduCam_Demo.cpp, however that is not the generic OpenCV way.

[ArduCAM]

@UberEclectic Our USB camera shield is not V4L2 compliance, we have cpp and python example code to open and view the camera , you can find from: https://github.com/ArduCAM/ArduCAM_USB_Camera_Shield/blob/master/RaspberryPi/Cpp/Streaming_demo/ArduCam_Demo.cpp Please note you might require to install the openCV.

[UberEclectic]

@ArduCAM Yes, I am aware. It was very difficult to integrate your camera into my OpenCV-based project, but I did achieve it in the space of a day. It seems like it should be possible for you to write an intermediary "driver" as an extension to OpenCV that "just works" with your cameras, but I wouldn't be as motivated to do that work as you might be. :-/

[ArduCAM]

Can you give me some hint how the intermediary driver should be, in terms of the interface or something like that, then we can work on that. Can you email to our email address admin@arducam.com , Thanks.

[UberEclectic]

@ArduCAM I expect it will take a clone + extension in the following area:

https://docs.opencv.org/master/dd/de7/group__videoio.html

[UberEclectic]

@ArduCAM Probably the easiest for you would be to turn your existing Streaming example into a listener for consumption as a URL-based stream provider. User would run your code to interface with the camera and make the stream available on a compatible protocol, then tell their OpenCV code to consume your stream via URL.

Read here:

https://docs.opencv.org/master/d8/dfe/classcv_1_1VideoCapture.html#ac4107fb146a762454a8a87715d9b7c96

[ArduCAM]

Thank you very much, we will look into this.

[UberEclectic]

Hey, @ArduCAM, I wanted to let you know that I am also looking into how to stand up a streaming server in place of readImage_thread().

To that end, can you tell me what would be the "native" video format from the camera? Is it closer to mjpeg, H.264, or another?

I want to avoid unnecessary transcoding / reprocessing, as much as possible. Just whatever is closest to the camera's native format and can be processed by OpenCV is what I intend to use. Thanks in advance.

[UberEclectic]

Hey, @ArduCAM, I found some more info on the original approach of extending OpenCV by linking your libraries. This may be simpler than making a generic RTP listener, although the streaming approach could ultimately mean more utility, since people could easily use your camera's as WebCams.

I may go back to trying to work with your libraries and OpenCV directly, but I a little worried that I do not have enough information to be successful. :S

Here is the additional info that I found:

https://docs.opencv.org/3.3.0/d0/da7/videoio_overview.html

[ArduCAM]

@UberEclectic The native video format depends on what kind of the camera module you will use. Different cameras support different output format like RAW, YUV, RGB, JPEG, our camera simpley capture whatever data comes from the sensor, you have to render the image on the backend. For openCV you can find that we have a function to convert the RAW image format to Mat format.

[UberEclectic]

Thanks, @ArduCAM do you have a document that indicates what native video format corresponds to each of your cameras? I have the Arducam CMOS MT9V022 1/3-Inch 0.36MP Monochrome Camera Module.

[ArduCAM]

Please check this one: http://www.arducam.com/docs/camera-breakout-board/0-36mp-mt9v034-mt9v022-global-shutter/