Open JoeAYeung opened 2 years ago
The gstreamer element is lacking some functionality at the moment. I hope that it will get resolved soon, but the existing contributors who worked towards adding the required support have not completed it, so it hasn't made it into libcamera.
Meanwhile, you can avoid that limitation by explicitly stating the required capabilities, or using a videoconvert element.
Try putting in the videoconverter which will help fix up missing caps,
gst-launch-1.0 libcamerasrc ! videoconvert ! xvimagesink !
or experiment with setting the required caps directly. In particular, it may specifically require colorimetry and framerate to be set explicitly:
gst-launch-1.0 libcamerasrc ! 'video/x-raw,width=1920,height=1080,colorimetry=bt709,framerate=30/1' ! autovideosink
or encode with something like:
gst-launch-1.0 libcamerasrc ! video/x-raw, colorimetry=bt709,format=NV12,interlace-mode=progressive,width=1280,height=720,framerate=30/1 ! videoconvert ! v4l2h264enc ! filesink location=captured.h264
I'm really keen to see the gstreamer element be more useful so let me know what you discover.
Thanks @kbingham !! videoconvert did the trick!!
Hi @kbingham ,
Have you seen this error message in the last command? How much GPU memory should I reserve?
../sys/v4l2/gstv4l2videoenc.c(828): gst_v4l2_video_enc_handle_frame (): /GstPipeline:pipeline0/v4l2h264enc:v4l2h264enc0: Maybe be due to not enough memory or failing driver
I believe the v4l2 h264 encoder will be providing memory from the CPU, not the GPU. So if you have a large gpu_mem=256, then it is taking memory /away/ from the pipeline.
Try reducing gpu_mem=256. I've also seen reports that you might find better functionality if you ensure the vc4-kms-v3d overlay is enabled:
https://github.com/ArduCAM/Arducam-Pivariety-V4L2-Driver/issues/14#issuecomment-1039518873
# Enable DRM VC4 V3D driver
dtoverlay=vc4-kms-v3d
Oh - and also if you try to remove the videoconvert element you'll get better performance, as that can cause CPU conversions to occur. But that is the part that currently requires getting the magic incantation for the gstreamer caps correct:
video/x-raw,colorimetry=bt709,format=NV12,interlace-mode=progressive,width=1280,height=720,framerate=30/1
is just an example, and may need more checking.
I believe the error above is not the actual error being faced. I haven't kept the link but on the rpi forums, there are topics about this (and I had the problem myself). Basically, IIRC, recent-ish kernels make it possible for gstreamer to try to encode using other H264 profiles but the hardware cannot actually do that or not at the corresponding bitrates.
The following caps helped me:
v4l2h264enc ! 'video/x-h264,level=(string)4'
The whole command is:
gst-launch-1.0 libcamerasrc ! video/x-raw, colorimetry=bt709,format=NV12,interlace-mode=progressive,width=1920,height=1080,framerate=30/1 ! v4l2h264enc ! 'video/x-h264,level=(string)4' ! filesink location=captured.h264
Well, except that my pi0 only goes up to 720p. Not sure I can get 1080p on that hardware (pi0 2 should might arrive on tomorrow), or maybe I did. I'm having contradicting memories. :D
Yes, - unfortunately - on the RPi - it is not sufficient to simply place the v4l2h264enc element without the level caps afterwards. Perhaps that can be fixed by RPi sometime or gstreamer, - but if it helps anyone, I've been developing on an IMX519 lately and testing the autofocus features in libcamera.
I use the following pipelines to stream the captured video from my RPi to my desktop
gst-launch-1.0 -vvv libcamerasrc ! video/x-raw,colorimetry=bt709,format=NV12,interlace-mode=progressive,width=1280,height=720,framerate=30/1 ! v4l2h264enc extra-controls=controls,video_bitrate_mode=0,video_bitrate=1000000,repeat_sequence_header=1 ! video/x-h264,profile=high,level=(string)4.2 ! h264parse ! rtph264pay ! udpsink host=monstersaurus port=11264
Note: 'monstersaurus' is the destination to send the video, which is my PC hostname.
You should not need to add a videoconvert element, and should avoid doing so - as that can cause software format conversions.
To receive the video on my PC I use
gst-launch-1.0 -v udpsrc port=11264 caps = 'application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96' ! rtph264depay ! decodebin ! videoconvert ! autovideosink
Thanks for the command-line, I never get these completely right. :D
I've edited my post to remove the videoconvert. I thought I didn't use it anymore actually. I guess I didn't notice because it doesn't change CPU usage and probably does nothing here. Still, better to avoid it.
Thanks for the autofocus work, that will be awesome!
By the way, I've gotten 1080p working on my rpi0 v1. I had to increase gpu_mem from 32 to 64. My /boot/config.txt now contains the following:
dtoverlay=vc4-kms-v3d,cma-32
max_framebuffers=1
gpu_mem=64
I am using RPi3 and followed instruction to install driver using the install script https://github.com/ArduCAM/Arducam-Pivariety-V4L2-Driver/releases/tag/install_script
Only
libcamera-still -t 0
workedOpening /dev/video0 with VLC did not give me any image. But more importantly, gstreamer doesn't work.
I noticed there are newer version of libcamera than what was referred to in the install_script in https://github.com/ArduCAM/Arducam-Pivariety-V4L2-Driver/releases
Should the install_script be updated? Or is my problem not related to libcamera version?
I am using RPi OS with 5.10.63-v7+ kernel
Below is the actual gstreamer error message