search

umlaeute / v4l2loopback

v4l2-loopback device
GNU General Public License v2.0
3.67k stars 523 forks source link

Minimal gstreamer H264 pipeline example. #271

Open janderholm opened 4 years ago

janderholm commented 4 years ago

I'm trying to write H264 to a v4l2loopback but am having trouble figuring out how to do it. This gstreamer pipeline works:

$ gst-launch-1.0 videotestsrc ! x264enc ! "video/x-h264,width=1280,height=720" ! decodebin ! videoconvert ! autovideosink

But when replacing the autovideosink with v4l2loopback it doesn't:

$ gst-launch-1.0 videotestsrc ! x264enc ! "video/x-h264,width=1280,height=720" ! v4l2sink device="/dev/video0"

Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
/GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc0.GstPad:src: caps = video/x-raw, framerate (fraction)30/1, width=(int)320, height=(int)240, format=(string)Y444, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, multiview-mode=(string)mono
ERROR: from element /GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc0: Internal data stream error.
Additional debug info:
gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc0:
streaming stopped, reason not-negotiated (-4)
ERROR: pipeline doesn't want to preroll.
Setting pipeline to NULL ...
Freeing pipeline ...

I've read open and closed issues and it seems that it might be possible? But there's no minimal working example that I could adapt to my particular use case. In the end I think I'm expecting v4l2-ctl --list-formats -d /dev/video0 to output an actual format like "H264 with start codes" or something. I'm guessing I'm missing missing some caps or elements to make things work.

derneue commented 4 years ago

I don't know if that would work. What should the v4l2loopback device do with an h264 encoded stream?

Can you test with:

gst-launch-1.0 videotestsrc ! v4l2sink device=/dev/video0

In another terminal you can read back with

gst-launch-1.0 v4l2src device=/dev/video0 ! autovideosink sync=false
zougloub commented 4 years ago

There are a bunch of open issues about using gstreamer with v4l2loopback, instead of dealing with them I wanted to see whether there was another way of getting a virtual H264 webcam.

When using ffmpeg (>= 4.3) to directly output h264 (I used -c:v libx264 -tune zerolatency -b:v 1600k -f v4l2 /dev/video10) there is continuous playback errors in mplayer and the video shows artifacts.

I managed to get something that's kind-of working (mplayer spews some errors for a few seconds, so the h264 packaging must not be ideal, but then the video doesn't show artifacts... there is noticeable latency also) with a crazy method using 2 phases, a first one where the video is sent uncompressed to a v4l2loopback device, and second one using https://github.com/mpromonet/v4l2tools like v4l2compress -f H264 -C 1600 /dev/video10 /dev/video11 to compress the video and send to a second v4l2loopback device.

My aim was to avoid using the browser's encoder, using a camera providing already-compressed data, but firefox&chrom* weren't happy with any of my attempts. Of course I haven't even checked whether they're supposed to work with an H264 camera.