OpenCV Video Capture Source

[jdye64]

In my current use case I spend a lot of time capturing images from IP cameras and processing those images in pipelines. Think home network security NVR with lots of bells and whistles. It would be nice to have a source where I could supply some simple information about my capture camera (IP, user, pass, channel, framerate, etc) and have Streamz connect to the camera using something like OpenCV and capture those images from the device and then emit them to the configured pipeline. In this manner I could setup complex pipelines for handling numerous different situations.

[CJ-Wright]

Are the images pushed into the pipeline or do you pull them from the camera at some rate?

[jdye64]

They are consumed from the camera at a constant user defined frame rate. Much the same way the Kafka source behaves with its poll interval.

[CJ-Wright]

Hmm, maybe we should generalize to a poll source that runs a function on an interval? I guess you could dump the image bytes to standard out and then use the from_process source, although that feels like a hack to me.

[jdye64]

So I have done a lot with opencv in C/C++ in the past. I just started looking at the python wrapper and its pretty awesome. In fact its got me for sure wanting to make a dedicated source but something more generic like "capture_video" the reason is with just a simple parameters it can support things like USB web cams, network IP cameras, videos from files (I know we already have from_file but this would format things into "frames" automatically), etc. So it seems quite handy and verbose to me. I'm more than happy to make it.

[jdye64]

Also I do like the idea of a generic "poll" source as well! In fact that was what I was trying to say I wanted https://github.com/python-streamz/streamz/issues/274 although poorly worded =)

[jtrevathan]

I agree that this would be useful. I am working with a very similar circumstance using opencv and pyueye (python wrapper for machine vision cameras that use the ueye API) and then streamz to handle the processing of the video feed before it is displayed. Right now I have it set up with a thread to poll the camera and emit frames in the stream. I think having a generic poll source in streamz would simplify handling of this.

[CJ-Wright]

PRs welcome!

[ingwinlu]

I did a PoC implementation at https://github.com/ingwinlu/streamz-opencv Feedback welcome, did not PR as I did not have time to look at how you do dependencies (like kafka) yet and did not simply want to pull opencv in.

Sadly can't seem to get good fps out of integrated webcam via opencv (far from the 30 or 25fps that it should be able to deliver).

[martindurant]

@CJ-Wright , did you have a chance to have a look at the POC? It would be reasonable to include such a thing (would not need to make openCV a dependency, and CI would need some sort of fake source, not a camera)

[CJ-Wright]

I think that would be great! I just looked at it now. Do you want to put it under the streamz org? Do you want to put it into streamz itself? Should we ship openCV as an optional dep?

[ingwinlu]

and CI would need some sort of fake source, not a camera

the opencv source could also be a small video with a few frames. that would integration test the streamz part of it nicely.

[martindurant]

That sounds perfect - presumably we can stream them at approximately the correct frame-rate!

Given that the code is short, I'd be happy to see it in the main library. The class should raise an informative exception if openCV is not installed, but no need to add to requirements. It should be installed for testing (but add pytest.importorskip to the test, so tests still run without)

[martindurant]

Did this idea get posted as code anywhere?