Open bugsyb opened 2 years ago
6by9
commented
2 years ago Totally unrelated to #1673 which is related to decode. It is a duplicate of https://github.com/raspberrypi/linux/issues/3974#issuecomment-791422239 - you haven't set the H264 level, and it'll default to level 1.0 (max 128x96 @ 30.9fps).
bugsyb
commented
2 years ago Thank you @6by9.
The test with videotestsrc worked:
gst-launch-1.0 videotestsrc ! v4l2h264enc ! 'video/x-h264,level=(string)3' ! fakesink
but am still struggling with getting v4l2src working (no crash anymore) and is most probably lack of my experience, but can't find.
Did check:
https://gitlab.freedesktop.org/gstreamer/gst-plugins-good/-/issues/849
And this workflow reported as working:
gst-launch-1.0 v4l2src device=/dev/video0 ! 'video/x-raw,framerate=30/1,format=UYVY' ! v4l2h264enc ! 'video/x-h264,level=(string)4' ! filesink location = test_video.h264
fails on me with:
ST_DEBUG=2 gst-launch-1.0 -vvvvv v4l2src device=/dev/video0 ! 'video/x-raw,framerate=30/1,format=UYVY' ! v4l2h264enc ! 'video/x-h264,level=(string)4' ! fakesink
Setting pipeline to PAUSED ...
0:00:00.049506042 688755 0x559dcc7640 WARN v4l2 gstv4l2object.c:4406:gst_v4l2_object_probe_caps:<v4l2h264enc0:src> Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Invalid argument
Pipeline is live and does not need PREROLL ...
0:00:00.050786122 688755 0x559dcc1920 WARN basesrc gstbasesrc.c:3127:gst_base_src_loop:<v4l2src0> error: Internal data stream error.
0:00:00.050833065 688755 0x559dcc1920 WARN basesrc gstbasesrc.c:3127:gst_base_src_loop:<v4l2src0> error: streaming stopped, reason not-negotiated (-4)
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
../libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
ERROR: pipeline doesn't want to preroll.
Setting pipeline to NULL ...
Freeing pipeline ...What am I doing wrong? One or two working example with v4l2src and v4l2h264enc would be of great use - if you could please kindly put together.
Thank you in advance.
6by9
commented
2 years ago What is your V4L2 source device?
What formats does it support? Output of v4l2-ctl --list-formats-ext and v4l2-ctl -V please.
I suspect you're now hitting the GStreamer v4l2*enc plugins advertising a limited set of colorimetry caps, as is being discussed in https://github.com/raspberrypi/firmware/issues/1673. My USB webcam advertises what GStreamer views as colorimetry=(string)2:4:16:1, which isn't a standard.
gst-launch-1.0 -vvvvv v4l2src device=/dev/video0 ! 'video/x-raw,width=640,height=480,framerate=5/1,format=YUY2' ! capssetter caps="video/x-h264, colorimetry=bt709" ! v4l2h264enc ! 'video/x-h264,level=(string)4' ! fakesink should work, but my webcam seems to be upsetting the USB bus and triggering a reset every time it starts up (reset high-speed USB device number 3 using xhci_hcd in the kernel logs).
bugsyb
commented
2 years ago Thank you. I did loads of reading (many nights as it is DIY project for me) and got lost with all these parameters, dependencies, etc. and it looks like some real issues. Thank you for your help.
Below are details, one is Pi cam2 and the other is usb cam (Microdia Webcam Vitade AF) marketed as "Wide-Angle 1080p UVC-Compliant USB Camera Module with Metal Case - Arducam" sold on The Pi Hut. Pi cam:
v4l2-ctl -d /dev/video0 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
Type: Video Capture
[0]: 'pBAA' (10-bit Bayer BGBG/GRGR Packed)
Size: Discrete 3280x2464
Size: Discrete 1920x1080
Size: Discrete 1640x1232
Size: Discrete 640x480
[1]: 'BG10' (10-bit Bayer BGBG/GRGR)
Size: Discrete 3280x2464
Size: Discrete 1920x1080
Size: Discrete 1640x1232
Size: Discrete 640x480
[2]: 'BA81' (8-bit Bayer BGBG/GRGR)
Size: Discrete 3280x2464
Size: Discrete 1920x1080
Size: Discrete 1640x1232
Size: Discrete 640x480v4l2-ctl -d /dev/video0 -V
Format Video Capture:
Width/Height : 1920/1080
Pixel Format : 'pBAA' (10-bit Bayer BGBG/GRGR Packed)
Field : None
Bytes per Line : 2400
Size Image : 2592000
Colorspace : sRGB
Transfer Function : sRGB
YCbCr/HSV Encoding: ITU-R 601
Quantization : Full Range
Flags :
USB cam:
v4l2-ctl -d /dev/video0 -V
Format Video Capture:
Width/Height : 1920/1080
Pixel Format : 'pBAA' (10-bit Bayer BGBG/GRGR Packed)
Field : None
Bytes per Line : 2400
Size Image : 2592000
Colorspace : sRGB
Transfer Function : sRGB
YCbCr/HSV Encoding: ITU-R 601
Quantization : Full Range
Flags : v4l2-ctl -d /dev/video2 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
Type: Video Capture
[0]: 'MJPG' (Motion-JPEG, compressed)
Size: Discrete 1920x1080
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 1280x720
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 640x480
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 320x240
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 1920x1080
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.033s (30.000 fps)
[1]: 'YUYV' (YUYV 4:2:2)
Size: Discrete 640x480
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 320x240
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 640x480
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.033s (30.000 fps)The Pi camera configured for libcamera needs libcamera to run in order to handle exposure, gain, white balance, lens shading, demosaicing, denoising, etc. The video encoder does not support Bayer formats. You need to be looking at libcamerasrc instead of v4l2src.
UVC (USB Video Class) should work fine, though may well need the capssetter override on the colorimetry. As I said my webcam seems to throw a reset each time it starts streaming, but that looks to be my camera as it fails on Pi4, CM4, and my x86_64 laptop (ie 3 different USB host controllers).
bugsyb
commented
2 years ago Thank you @6by9.
Managed to get pipeline which didn't error out, but v4l2h264 doesn't seem to respect bit rate settings. At least I can't manage to get that done. Used video_bitrate_mode=0 and video_bitrate=25000, where _video_bitrate_mode has been tested with 0 & 1 and with additional settings of video_bitrate. Regardless of settings am getting ~5Mb/s even with completely black content on the screen. Seems like the v4l2h264enc is not that good. probably due to limitations of bmc. vp8 (cpu based sends same content at ~100kb/s). It has some hiccups due to cpu, but is of a huge difference.
Is there a chance to have a control over bitrate/compression flags/levels?
To repeat some success and for others to get a working pipeline (below what worked for me):
gst-launch-1.0 -vvv v4l2src device=/dev/video2 ! 'image/jpeg,framerate=30/1,width=1920,height=1080'! v4l2jpegdec ! videoconvert ! v4l2h264enc ! 'video/x-h264,level=(string)4,c' ! h264parse ! rtph264pay ! udpsinkFor a repeatable test:
gst-launch-1.0 -vvv videotestsrc num-buffers=300 ! video/x-raw,framerate=30/1,width=1280,height=720,format=I420 ! capssetter caps="video/x-h264, colorimetry=bt709" ! v4l2h264enc extra-controls="controls,video_bitrate=100000,video_bitrate_mode=1" ! 'video/x-h264,level=(string)4' ! filesink location=100k.h264
gst-launch-1.0 -vvv videotestsrc num-buffers=300 ! video/x-raw,framerate=30/1,width=1280,height=720,format=I420 ! capssetter caps="video/x-h264, colorimetry=bt709" ! v4l2h264enc extra-controls="controls,video_bitrate=1000000,video_bitrate_mode=1" ! 'video/x-h264,level=(string)4' ! filesink location=1M.h264
gst-launch-1.0 -vvv videotestsrc num-buffers=300 ! video/x-raw,framerate=30/1,width=1280,height=720,format=I420 ! capssetter caps="video/x-h264, colorimetry=bt709" ! v4l2h264enc extra-controls="controls,video_bitrate=10000000,video_bitrate_mode=1" ! 'video/x-h264,level=(string)4' ! filesink location=10M.h264ls -l *.h264
-rw-r--r-- 1 pi pi 498464 Mar 9 14:34 100k.h264
-rw-r--r-- 1 pi pi 1251538 Mar 9 14:34 1M.h264
-rw-r--r-- 1 pi pi 9303021 Mar 9 14:35 10M.h264Repeat with video_bitrate_mode=0 and I get
-rw-r--r-- 1 pi pi 314284 Mar 9 14:36 100k.h264
-rw-r--r-- 1 pi pi 1251024 Mar 9 14:36 1M.h264
-rw-r--r-- 1 pi pi 12500280 Mar 9 14:36 10M.h2641MBit/s is 125kbytes/sec, so recording for 300 frames at 30fps is 10 seconds. Getting an output file pretty much bang on 1250kB is pretty reasonable rate control in my book.
6by9
commented
2 years ago Minor caveat: those were done against Gstreamer 1.21 main branch instead of the stock Raspberry Pi OS 1.18. I don't believe anything significant has changed with v4l2h264enc between those two points.
talksik
commented
1 year ago Thanks for the hints everyone. For any newcomers, I was able to solve this with the following guide that I put together myself after days of experimentation.
The v4l2src element shoots an error about not enough memory allocated and it errors out at that first element of the pipeline.
I was able to solve it with this:
$ v4l2-ctl --list-formats
ioctl: VIDIOC_ENUM_FMT
Type: Video Capture
[0]: 'YU12' (Planar YUV 4:2:0)
[1]: 'YUYV' (YUYV 4:2:2)
[2]: 'RGB3' (24-bit RGB 8-8-8)
[3]: 'JPEG' (JFIF JPEG, compressed)
[4]: 'H264' (H.264, compressed)
[5]: 'MJPG' (Motion-JPEG, compressed)
[6]: 'YVYU' (YVYU 4:2:2)
[7]: 'VYUY' (VYUY 4:2:2)
[8]: 'UYVY' (UYVY 4:2:2)
[9]: 'NV12' (Y/UV 4:2:0)
[10]: 'BGR3' (24-bit BGR 8-8-8)
[11]: 'YV12' (Planar YVU 4:2:0)
[12]: 'NV21' (Y/VU 4:2:0)
[13]: 'RX24' (32-bit XBGR 8-8-8-8)
$ v4l2-ctl --set-fmt-video=width=640,height=480,pixelformat=9
$ v4l2-ctl -V
Format Video Capture:
Width/Height : 640/480
Pixel Format : 'NV12' (Y/UV 4:2:0)
Field : None
Bytes per Line : 640
Size Image : 460800
Colorspace : SMPTE 170M
Transfer Function : Default (maps to Rec. 709)
YCbCr/HSV Encoding: Default (maps to ITU-R 601)
Quantization : Default (maps to Limited Range)
Flags :And now this works:
$ gst-launch-1.0 -e v4l2src ! video/x-raw,width=640,height=480,framerate=30/1 ! videoconvert ! fakesinkNotice how the capabilities match those which we set with v4l2-ctl.
This works for test video:
$ gst-launch-1.0 videotestsrc ! v4l2h264enc ! 'video/x-h264,level=(string)3' ! fakesinkEncoding from camera:
$ gst-launch-1.0 -e v4l2src ! video/x-raw,width=640,height=480,framerate=30/1,format=NV12 ! v4l2h264enc ! 'video/x-h264,level=(string)4' ! fakesink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Redistribute latency...
0:00:10.5 / 99:99:99.The format much be something that gst-inspect-1.0 v4l2src has, AS WELL AS something that gst-inspect-1.0 v4l2h264enc accepts. In my system, I chose to go with NV12.
$ v4l2-ctl -V
Format Video Capture:
Width/Height : 640/480
Pixel Format : 'NV12' (Y/UV 4:2:0)
Field : None
Bytes per Line : 640
Size Image : 460800
Colorspace : SMPTE 170M
Transfer Function : Default (maps to Rec. 709)
YCbCr/HSV Encoding: Default (maps to ITU-R 601)
Quantization : Default (maps to Limited Range)
Flags :
Every time trying to run use v4l2h264enc via gstreamer, below kernel crash is visible:
5.10.63-v8+ #1459 SMP PREEMPT Wed Oct 6 16:42:49 BST 2021 aarch64 GNU/LinuxBut running this, works fine
PARAM=" --width=1920 --heigh=1080 --codec yuv420 --framerate 15" libcamera-vid -t 0 $PARAM -o - | gst-launch-1.0 -vvvv fdsrc fd=0 ! rawvideoparse use-sink-caps=false width=1920 height=1080 format=i420 ! v4l2convert ! v4l2h264enc extra-controls="controls,video_bitrate_mode=0,video_bitrate=1000000,repeat_sequence_header=1" ! 'video/x-h264,profile=high,level=(string)4.2' ! h264parse ! rtph264pay ! udpsink host=myhost port=5001Could it be related to: https://github.com/raspberrypi/firmware/issues/1673 ?