6by9
commented
3 years ago I believe it is to do with job_ready only allowing a job to proceed if both src and dest buffers are available, in specific in static int job_ready(void *priv) in bcm2835-v4l2-codec.c
job_ready returns 0 if there are BOTH src or dest have NO buffers, therefore returns 1 if either have buffers to process.
if (!v4l2_m2m_num_src_bufs_ready(ctx->fh.m2m_ctx) &&
!v4l2_m2m_num_dst_bufs_ready(ctx->fh.m2m_ctx))
return 0;
return 1;It's also why we call v4l2_m2mset[src|dest]_buffered to tell the m2m framework that we will accept buffers on the 2 queues independently.
The error message no space in FIFO for headers means what it says, although it should apply for any encoded output.
The FIFO is created with 200 entries, so at a default Intra-IDR period of 60 frames I would have expected that to be exhausted at around 12000 frames if it were a simple leak every time we added the extra headers. V4L2 does differ from the MMAL/IL clients in that it sets a flag to request that the header bytes are prepended to the I/IDR frame, rather than sent as separate buffers.
Rereading the V4L2 spec, I do see V4L2_CID_MPEG_VIDEO_HEADER_MODE, which FFmpeg sets to V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE to ask for the headers separate from the I/IDR frame. That one should be fairly simple to add as a control.
The other possibility I can think of is that the codec has produced a frame that exceeds the V4L2 buffer size. This is one of the ugly parts of the V4L2 codec spec that it mandates that a frame must fit into a single CAPTURE buffer, but you can never guarantee the max size of an encoded frame when doing a 1-pass encode. The spec does state
If a CAPTURE buffer is too small then it is just returned with the V4L2_BUF_FLAG_ERROR flag set. More work is needed to detect that this error occurred because the buffer was too small, and to provide support to free existing buffers that were too small.which we don't currently do, and will return a second CAPTURE buffer with the remaining part of the encoded frame. I don't know if that will upset FFmpeg if it is relying on a 1-in, 1-out behaviour. Fixing that one could be trickier.
Yes #4419 applies to the codec too - we use the ISP hardware block for the conversion on both the input of the encoder and the output of decoder. I'll sort the patch for that now.
apu727
Is this the right place for my bug report? I believe it is associated with the v4l encode device.
Describe the bug
video_encode:21:RIL: no space in FIFO for headersis printed insudo vcdbg log msgwithvcgencmd set_logging level=0x400100C0while encoding withREPEAT_SEQ_HEADERset to 1.The error message does not appear initially, but consistently after ~20000 frames encoded. The timing of when it appears doesn't appear to be too spread out The error message also does not appear if
REPEAT_SEQ_HEADER=0While using ffmpeg in code, frame write and packet read also became inconsistent after a while, which I traced back to all(16) output (data going to driver) buffers being in driver after a certain period which coincided with the error messages being produced. There were few outstanding capture (from driver) buffers, i.e the problem was not lack of capture buffers in driver.
Note: I am using ffmpeg only to generate and encode the video, I do not believe it is a problem with ffmpeg.
To reproduce get ffmpeg-4.4 release modify v4l2_m2m_enc.c to include and set
REPEAT_SEQ_HEADERto 1. as seen in this patch which was found: here build ffmpeg (any compile flags should be good I have attached the usedffmpeg -version)run
vcgencmd set_logging level=0x400100C0runffmpeg -re -loglevel info -f lavfi -i color=size=1408x1086:rate=30 -codec h264_v4l2m2m -pix_fmt yuv420p -b:v 8M -f null -runsudo vcdbg log msgand eventually the errors start appearingffmpeg_version.txt
Expected behaviour No error messages and no backlog of output(to driver) buffers
Actual behaviour Error messages and all output buffers stored in driver
System https://pastebin.com/37j7k6E2
I have removed dmesg and vcdbg from raspinfo as i will attach the relevant parts below. Let me know if you need this too (it is the same I believe)
Logs I have created logs of
dmesgandvcdbg log msgthey are named good/bad for when REPEAT_SEQ_HEADER is enabled/disabled respectively. For dmesgecho Y > /sys/module/v4l2_mem2mem/parameters/debugwas set. For 'bad' vcdbg i have created a sample of the logs at ~5000 frame intervalsdmesgbad.txt dmesggood.txt vcdbgloggood.txt vcdbglogbad5000.txt vcdbglogbad10000.txt vcdbglogbad15000.txt vcdbglogbad20000.txt vcdbglogbad25000.txt
Additional context I believe it is to do with job_ready only allowing a job to proceed if both src and dest buffers are available, in specific in
static int job_ready(void *priv)inbcm2835-v4l2-codec.cWhen testing it in code I had a breakpoint set for if all output buffers fill. When this breakpoint was hit
video_encode:21:RIL: no space in FIFO for headerswas printed continuously while the program was paused with no intermediate messages(vcgencmd set_logging level=0x400100**4**0) I have logs of that too if needed.The weird size and static colour are to replicate the images I was using when I noticed it. The width was also chosen to be 64 byte aligned. see https://github.com/raspberrypi/linux/pull/4419 which I believe affects bcm2835-v4l2-codec too (seperate issue)