6by9
commented
4 years ago Thank you for a fairly comprehensive bug report.
It sounds like you're just running out of CMA heap, in which case the VideoCore side logs won't help much. If neither vc4-fkms-v3d nor vc4-kms-v3d are loaded, then the default CMA heap is only around 16MB in size. You won't fit many 1080p YU12 frames in that after the other allocations. With either of those overlays it normally defaults to 256MB, which should be plenty. If you grep the normal dmesg output, the size of the CMA heap is normally listed within the first few lines during boot.
Please note that during format renegotiation the buffers are not immediately released. They are retained until the VideoCore side has confirmed that it really has finished with them. This shouldn't take any significant time, but may leave a window where both sets of buffers are allocated.
I'll have a look when I'm back in the office next week.
eocanha
RogerHardiman
cadubentzen
Bug description and context
I'm using a customized version of Buildroot with a custom defconfig on a Raspberry Pi 3B+ to try play YouTube videos on the WPE WebKit web engine using Media Source Extensions (not functional in the provided configuration, but keep reading).
WebKit uses GStreamer as its multimedia framework in this configuration, Wayland for graphics output and the V4L2 memory-to-memory video codecs instead of the OpenMAX ones. YouTube uses MSE to implement adaptive streaming (change video resolution/quality on the fly) so, at some point, the quality of the H264 video stream changes from 640x360 to 1920x768. GStreamer handles the changes properly by flushing the decoder, deallocating the input/output buffers from its buffer pools, renegotiating the capabilities with the V4L2 decoder and trying to allocate 2 new input buffers (H264 data) and 2 new output buffers (FullHD raw video data). This last allocation fails and the errors stop the pipeline and reach the browser.
At this point I got rid of the browser and could reproduce the issue just by using a gst-launch-1.0 command to play a video-quality-change.mp4 file generated by concatenating the 640x360 H264 header, the first data fragment, the 1920x1080 header, and a data fragment in that new resolution (exactly the same data that YouTube fetches and MSE provides to the pipeline). Note that the file is properly played by gst-play-1.0/gst-launch-1.0 on builds that use the OpenMAX GStreamer elements instead of the V4L2 ones. Also note that on desktop you can only properly play the file in GStreamer if you force "push mode" (sequential access to the file instead of random access):
As mentioned in https://www.raspberrypi.org/forums/viewtopic.php?t=220640#p1586156 , I tried to debug what's happening on the VideoCore 4 side but couldn't use vcgencmd and vcdbg (let's call them "vc tools") due to the vc4drmfb kernel framebuffer module not exposing the required ioctls that allow the vc tools to talk to videocore. To simplify debugging, I disabled dtoverlay=vc4-kms-v3d. That made the vc tools work again but left me without Wayland nor KMS to display the output. Anyway, at this point I could continue debugging just by using GStreamer's fakesink (basically, no output) and examining the GStreamer/kernel/vctools logs of the behaviour of the codecs. This is the scenario to be found if using the buildroot branch and defconfig provided above.
In this situation, the logs show how GStreamer renegotiates the new resolution with the driver, allocates a new buffer pool but only gets 1 of the 2 output buffers that were requested (only relevant lines pasted):
The kernel can only allocate one of the two requested buffers. Cma_alloc (whatever that is) fails for the second one:
At this point I hoped that the vcdbg logs (after "vcgencmd set_logging level=2147483647") would be of some help to explain why the allocation failed, but they aren't:
Steps to reproduce
1A. Short way: Format an sdcard with an vfat (boot) partition and an ext4 (root) partition and uncompress the boot.tar.gz and root.tar.gz archives. 1B. Long way: Clone the https://github.com/eocanha/buildroot repository and check out the upstream-v4l-codecs-newkernel branch. From the buildroot directory, copy configs/raspberrypi3_wpe_wayland_v4l2_debug_experimental_defconfig as .config.
make menuconfig, save it,make. Copyoutput/images/*.dtb,output/images/zImage,output/images/rpi-firmware/*to the boot partition of the sdcard. Uncompressoutput/images/rootfs.tarin the root partition of the sdcard. Copyoutput/build/rpi-firmware-0c01dbefba45a08c47f8538d5a071a0fba6b7e83/hardfptoroot/in the root partition of the sdcard. Also copy theroot/script.shlauncher script and the video-quality-change.mp4 video from the root.tar.gz archive of the 1A step toroot/in the sdcard.Expected behaviour
There should be no errors in the logs, which should show lines logging produced frames after the quality change. No images will be displayed in any case with this setup (fakesink), so don't expect them.
Actual behaviour
The buffer pool negotiation errors explained in the description of this bug are shown in the logs.
System