search

hybridgroup / gocv

Go package for computer vision using OpenCV 4 and beyond. Includes support for DNN, CUDA, and OpenCV Contrib.
https://gocv.io
Other
6.63k stars 863 forks source link

Capture WebRTC #433

Open xrayid opened 5 years ago

xrayid commented 5 years ago

Hello! Can I capture WebRTC stream using this lib?

deadprogram commented 5 years ago

Hi @xrayid you should be able to do so by using gstreamer as the video input, but I have not done it yet myself.

https://blog.nirbheek.in/2018/02/gstreamer-webrtc.html

Please let us know what you find out if you investigate further.

xrayid commented 5 years ago

Thx @deadprogram ! I will try

chainius commented 5 years ago

Hi,

did you try to use this lib with WebRTC through GStreamer ? I'm also checking how to do it.

Sean-Der commented 5 years ago

There is a Pure Go implementation of WebRTC Pion WebRTC it has no C dependencies so should be really easy to use (NB I am the author)

I am working on getting this wired up now. I have a stream of VP8/H264, I don't know gocv/opencv that well but don't see an immediate way to consume an encoded video stream. Does anyone know if I can, or do I need to use libav?

@deadprogram I think having WebRTC + OpenCV would be huge for the Go community. If you could say Build a computer vision backend in 10 minutes that all platforms/browsers can send/recv against people would be really excited.

Would also be great if people wanted to capture from a weak device but then send to a more powerful server (WebRTC can do all the NAT hole punching and encryption for them)

deadprogram commented 5 years ago

Most people use gstreamer, which is often built-in to OpenCV. I have not used it with WebRTC.

You could do that same thing by getting the frame some other way, then turning that into a Mat for processing. An example of that pattern can be found in https://github.com/hybridgroup/gobot/blob/master/examples/tello_opencv.go you'd substitute the WebRTC video for the Tello drones UDP feed.

Sean-Der commented 5 years ago

Hey @deadprogram

Done! https://github.com/pion/example-webrtc-applications/tree/master/gocv-receive

GStreamer is fantastic, but the Go <-> C experience is painful (IMO) I have used GStreamer in NodeJS and Go applications, and once I finally get it all working I am afraid to touch it ever again :)

Sean-Der commented 5 years ago

@deadprogram @xrayid @Skyhark-Projects have you had a chance to try this out yet?

Anything I should change/improve?