Streaming other sources other than the webcam: OpenCV

[ignacioavellino]

Hi!

This is just a placeholder in case anyone can enlighten me on sending media other than the webcam, ideally from frames processed using OpenCV

My guess is that I do not stand alone on this. Sometimes we want to process the webcam image using OpenCV before sending it, so after I have my nice Mat frames in my while loop, I don't know how to plug them to OpenWebRTC in order to send them

Any help is much appreciated ^^

Thanks in advance,

—N

[superdump]

What memory is used to store your OpenCV frames? System memory or graphics memory or something else?

[richardmitic]

OpenCV stores frames in system memory by default. There are options for hardware acceleration but they're not used so much.

[sdroege]

This could be relatively simple solved by https://github.com/EricssonResearch/openwebrtc/issues/221 and using e.g. an appsrc.

Alternatively it might also make sense to have appsrc/appsink-style MediaSources/Renderers, where application can pass in data directly or get data from a callback.

[ignacioavellino]

Hey,

@superdump , frames are stored on memory as @richardmitic pointed out @sdroege , I'm not an expert in this subject, but your alternative sounds good, having a callback where the application can pass data

[richardmitic]

@ignacioavellino, bear in the mid that in OpenCV the pixel data is not necessarily contiguous in memory. In your app you'd have to either assure that it is contiguous or possibly work some gst_buffer magic to avoid expensive copy operations. It should work though. :)

[superdump]

@sdroege that's part of my plan. Widely-used custom sources/renderers gradually get merged back in. appsrc/appsink are obvious candidates.