Bullseye's VLC Won't Play libcamera-vid Video Files Properly

[ghalfacree]

The version of VLC currently provided in Raspberry Pi OS "Bullseye" does not appear able to play the H.264 files created by libcamera-vid.

Reproduction

libcamera-vid -t 10000 -o test.h264
vlc test.h264

Expected result

The video file, a timecode-free header-free VGA-resolution H.264 bitstream, plays smoothly in the VLC window which opens.

Actual result

VLC opens and begins to play the video at what appears to be half-speed. Soon, more frames are missing - resulting in glitching and skipping.

The same video plays properly - once the file manager is told it's a video - in VLC on an Ubuntu 20.04.3 desktop.

Versions

32-bit Raspberry Pi OS Bullseye Desktop installed via Imager from the image dated 30th October 2021 then fully updated, with no other changes beyond enabling SSH, running from a 16GB microSD card on a Raspberry Pi 4 Model B 8GB.

Linux raspberrypi 5.10.63-v7l+ #1488 SMP Thu Nov 18 16:15:28 GMT 2021 armv7l GNU/Linux.

VLC media player 3.0.16 Vetinari (revision 1.0.6-1682-g88158c836) VLC version 3.0.16 Vetinari (1.0.6-1682-g88158c836) Compiled by pi on serge-testpi (Nov 10 2021 15:40:45) Compiler: gcc version 10.2.1 20210110 (Raspbian 10.2.1-6+rpi1)

libcamera-apps build: 2a38ae93f143 21-10-2021 (14:49:21) libcamera build: v0.0.0+3156-f4070274

Supporting information

A user reported what appears to be the same issue on the 5th of December in the Bullseye bug thread on the Raspberry Pi Forum; no follow-up appears in the thread.

A sample test.h264 file and a video capture demonstrating the problem can be downloaded here; ignore the interlacing artefacts in the video capture, that's simply the default output the Raspberry Pi picks when it's connected to my HDMI capture card.

Using VLC to play back libcamera-vid files is the officially recommended approach and is also the default association if you double-click on the recording in the file manager, thus should be expected to work out-the-box.

Please let me know if you need any additional information.

[XECDesign]

I don't think VLC can play an h264 bitstream well without a container

@davidplowman, the documentation probably shouldn't mention playing the .h264 file directly, unless I'm missing something.

[ghalfacree]

I don't think VLC can play an h264 bitstream well without a container

The same file plays perfectly in VLC 3.0.9.2 on a 64-bit Ubuntu 20.04.3 desktop.

[XECDesign]

@jc-kynesim, would you be able to confirm that our version of VLC isn't expected to play h264 bitstreams directly like that?

[popcornmix]

Just tried this and test.h264 did play for me using (desktop) VLC on bullseye (Pi 4)

[davidplowman]

For me it plays on my Pi, but badly (dropping or skipping frames). Interestingly it plays badly on my PC too (Ubuntu 21.10). Both claim to be 3.0.16 Vetinari of some sort.

And on both platforms, ffplay does not exhibit any problems at all. Well, there's some strange screen blanking on the Pi when it starts/stops, but playback itself it perfect.

[ghalfacree]

Interestingly it plays badly on my PC too (Ubuntu 21.10). Both claim to be 3.0.16 Vetinari of some sort.

Huh - it's fine on 3.0.9.2 on my desktop. I wonder if there's some kind of regression upstream?

[popcornmix]

Played okay for me on PC Ubuntu 3.0.9.2.

[jc-kynesim]

I just looked at this (sorry about the enormous delay) and it found a bug in ffmpeg v4l2 stateful decode for me :-) but just testing with a current VLC (which uses mmal) seems to work fine for me - is there still believed to be a bug here?

[davidplowman]

sudo apt install vlc tells me I have the latest, so does that mean I'm running the right thing? It claims to be "3.0.16-1+rpi1+rpt2". Anyway, this version is still very broken in that it skips and drops frames creating jumps in the video. ffplay seems to have no trouble (presumably using ffmpeg as well).

[jc-kynesim]

Ah OK I now see what you mean - I've got to guess that the problem is the lack of timestamps is confusing something (presumably decode & display are trying to run at different rates). I'll have a look, but given that my PC desktop version does the same as the Pi (which is why I thought it was good) the issue isn't in our patches.

[davidplowman]

Yes, I think that's correct. The findings above would suggest that upstream vlc got broken sometime between 3.0.9 and 3.0.16, which is a bit unfortunate because we've rather relied on it in the past for demos and documentation.

[davidplowman]

Hi, Ok, I understand the issue is a VLC issue and not a libcamera-vid one. Am I right? / Thank you

Yes, that's correct.

[jc-kynesim]

Before I blame it all on VLC upstream there may be mitigating circumstances - what frame rate are you expecting? Looks like the ES has it coded @ 30Hz (from mediainfo - I haven't yet taken the stream apart to check)

[davidplowman]

libcamera-vid records at 30fps by default (you can change that), though so far as I know there's no timing information at all in the elementary stream (is that right?). I agree it looks like vlc tries to play at 15fps but the display-on-screen and read-from-file bits don't seem to co-operate, dropping frames on the floor. I think I did try some kind of "set the framerate to 30" option but I couldn't see any difference.

[jc-kynesim]

The issue is that the PTS/DTS faking code seems to think that DTS is running at 30Hz but PTS at 15Hz though there are points where it sets PTS=DTS - this, unsurprisingly, causes confusion. The code is all a bit labyrinthine so I haven't found where it goes wrong - I just know what it does

[ghalfacree]

Just to confirm, this is still broken in Bookworm (and Bullseye) as of the latest beta. Same behaviour: videos recorded without timestamps do not play back properly in VLC.

[jc-kynesim]

I'll look at this again - I thought I'd fixed it - maybe it was just fixed for some cases and not others

[ghalfacree]

Unfortunately, it's definitely still failing on a fully-updated Bookworm, as of yesterday afternoon: the attached video should show my hand entering the frame and waving smoothly, but instead it suddenly leaps into the shot in a mess of encoding blocks.

Let me know if you want me to test anything, though I'm not doing anything special - that's just the output of a libcamera-vid -t 10000 -o test.h264.

https://github.com/RPi-Distro/vlc/assets/1130041/575d6786-cbb4-4878-931c-f6b760763b24