Configuration fails on Raspbian buster with mmal enabled ffmpeg

[gy-lehel]

mpv version and platform

v0.29.1, and master Running on RPi 3B+ running buster

Reproduction steps

Using the builder script echo --enable-mmal >> ffmpeg_options ./use-mpv-release ./use-ffmpeg-release ./update ./rebuild -j4

Expected behavior

Successful configuration and compile

Actual behavior

mpv fails to configure

Log file

Sample files

[gy-lehel]

0001-6838-Fix-rpi-compile-issues.patch.txt

[daryltucker]

@gy-lehel thank you for providing this patch. It worked to allow compilation of mpv on Raspberry Pi 4 B against 723fd02919bb3a1249d0566a70cc4d448a1e8ae6.

../video/out/opengl/hwdec_vaegl.c:70:123: error: unknown type name ‘EGLuint64KHR’; did you mean ‘GLuint64’?

# System
4.19.60-1-ARCH, armv7h, Arch Linux

# Env
export PATH="$PATH:/opt/vc/sbin:/opt/vc/bin"
export PKG_CONFIG_PATH="/opt/vc/lib/pkgconfig:/usr/lib/pkgconfig:/usr/share/lib/pkgconfig"
export CPATH="/opt/vc/include:/usr/local/include:/usr/include"
export LD_LIBRARY_PATH="/opt/vc/lib:/usr/local/lib:/usr/lib"

# FFmpeg
./configure --arch=armel --target-os=linux --enable-gpl --enable-omx --enable-omx-rpi --enable-nonfree --enable-mmal --enable-openssl --enable-shared --enable-static

# mpv
./waf clean ; ./waf configure --prefix=/usr/local && ./waf build -j4

[CounterPillow]

@daryltucker why are you building against an old commit, and then reporting when it fails on a source file that no longer even exists in master?

[daryltucker]

Thanks. I specified because I didn't realize I was not on a recent commit. Thank you for mentioning. I thought that 723fd02919bb3a1249d0566a70cc4d448a1e8ae6 was the most recent, 0.29.1 release.

Edit: I checked out v0.29.1, which I believed to be the latest stable version of mpv, and was able to compile after finding @gy-lehel's suggestion. I wanted to help provide confirmation and additional details. Doesn't mpv-build have an option to lock into ffmpeg and mpv stable versions, and if so, I thought that this could be why this error is being reported and seen now in 2019. If not, should the release tags not be trusted, or is there some document I could read to help me understand where to find stable branches or tags?

[CounterPillow]

mpv doesn't really backport fixes to releases because the only reason releases are made is so that distros have something to package. If there is a bug it's best to test against git master to see if it still exists. In this case, the part you're having a compilation failure in no longer exists in that form in git master.

[Akemi]

already fixed apparently.

[gy-lehel]

Nope, not fixed, since there was no commit fixing it.

[ghost]

The proposed fix is incorrect. Also FFmpeg needs to export the libs it requires in the .pc file. The vaegl part is also incorrect since while it "fixes" compilation, the resulting code is nonsense. Anyway, nobody around here cares about rpi.

[gy-lehel]

Ok, I see but then please some of the insight on how to do it "properly", so that it does not become "nonsense". What I did is make it compile with buster and rpi4, while keeping backwards compatibility with wheezy, rpi3 and rpi2.

[gy-lehel]

0001-Fix-Raspbian-Buster-build-failures.patch.txt

I think I see what you mean, actually the 3rd layer was not defined at all, but the code still tried to refer to it. As far as I can see ffmpeg does it pretty well, the issue is with the case when mmal support is added into the game, then the configuration scripts start to miss out libraries.

[ghost]

I don't know why there's a 4th layer, but a commit in git master should make it build anyway. FFmpeg is buggy if it doesn't link on its own. Can't you see how wrong it is to add an exception for an obscure platform (and honestly utterly shitty) like RPI to this part of configure is? FFmpeg supports multiple dozen of external libs, and none need additional code at the API user site just to not break. By the way, it was me who regrettably added RPI support to FFmpeg. I know the RPI part in FFmpeg's configure code doesn't really make sense. I'm not contributing to FFmpeg anymore, nor does anyone in FFmpeg care about RPI, so you need to fix it yourself.

[ghost]

Oh, and the x11egl.c change: GLX will have to suffice if RPI doesn't support EGL 1.5. Or use open source drivers. Have they fixed their awful mess with "native" EGL conflicting with Mesa's? What a terrible platform.

[gy-lehel]

You bet, they did not! The entire thing is in a separate lib, the signatures messing up everything. And if one takes in mmal, then their implementation is included, that why the "fix"...

[ghost]

Wasn't there something about Mesa becoming pretty good on RPI? Maybe you can just use Mesa and disregard the BCM EGL, since MMAL is technically separate. Unless it can't be separated. Also I think the longterm plan was to get rid of MMAL and use V4L decoding? There might be light at the end of the tunnel, but you still have to dig yourself out of the collapsed section.

[gy-lehel]

Well, the problem seems to be, that mess is still part of this world. If one compiles ffmpeg with mmal support, the latter one is packaged together with the BCM EGL. No way of getting rid of it in a clean way. When ffmpeg is compiled this way, it won't be able to find mmal, since that is put into /opt... BCM EGL is completely broken for RPi4, the examples do not work at all, even with latest source pulled. As a result mpv falls back to xv, with abysmal performance. I'll try to give it a try without mmal, though the aim would be to use the HW acceleration... Basically the first (broken as you pointed out) patch will make it work on the RPi3+Buster combo, Rpi4 needs modifications, which I am afraid I am lacking the knowledge to do.

[ghost]

Ah, there I have to say again: not our problem.

There's a glvnd EGL library now (I think), which would them in theory allow to solve this.

[ivanich]

I'll try to give it a try without mmal, though the aim would be to use the HW acceleration...

I was trying to achieve the same and ended up with semi-working HW accel with mpv in Arch linux on RPI4, so the steps I did:

• build ffmpeg-mmal from aur(it simply enables mmal, omx, omx-rpi in configure script)
• added hwdec=mmal-copy to the mpv.conf (hwdec=mmal won't work)
• used vanilla mpv from archlinux repos(mpv-rpi from aur doesn't work)

Now I have 45-65 cpu load with this config and 1080 videos, before it was 150-200 cpu.

[ghost]

If it only fails on mpv's need for EGL in vo_rpi, maybe we could just revert the change that made it use GL for OSD. (Or make it work with Mesa's GL, if transparency and layering can be made work.)

Needs someone to develop this, though.

[gy-lehel]

I had some time to dig into this, here are my findings: With buster, raspbian started to deliver a standards compliant EGL implementation, and the old brcm (/opt/vc) in parallel. The root of the problem is, that if one takes in the mmal libraries one takes the brcm ones, too, which are trash for RPi4. I managed to compile everything well finally, by creating symlinks to the mmal package files, thus avoiding the inclusion of brcm EGL libraries, and adding --disable-rpi to mpv configuration options. I ended up with binaries that use gpu/GL for output had 30-32% CPU load for a full HD video, which is more-or-less the same performance as the mpv that is delivered as a package in buster, though I have to confirm it by running that on a system, that has not been messed so much as this one. :)

[NoSuck]

So no performance gains? That doesn't sound right.

I am unable to do a comparison for mpv, as I have yet to successfully build it. In the process of trying, however, I did manage to build ffmpeg with MMAL support (thanks to mpv-build). On my Pi 4 running Raspbian 10, MMAL uses 64～74% less CPU, which is wonderful. Here is how I tested:

uS=30 ; ./ffplay -fs -vcodec h264      "$VIDEO" & s="$!" ; mapfile -t < <( for (( u = 0 ; u < uS ; u++ )) ; do sleep 1 ; ps -o %cpu= $s ; done ) ; { printf '(' ; printf "%s+" "${MAPFILE[@]}" ; printf "0)/$uS\n" ; } | bc -l
uS=30 ; ./ffplay -fs -vcodec h264_mmal "$VIDEO" & s="$!" ; mapfile -t < <( for (( u = 0 ; u < uS ; u++ )) ; do sleep 1 ; ps -o %cpu= $s ; done ) ; { printf '(' ; printf "%s+" "${MAPFILE[@]}" ; printf "0)/$uS\n" ; } | bc -l

Hope you don't mind bash one-liners ;)

[NoSuck]

Hold the phone.

The previously stated performance gains were observed while playing back a video of relatively low bit rate (622 kb/s). The video at /opt/vc/src/hello_pi/hello_video/test.h264, meanwhile, has a much higher bit rate (~17,314 kb/s). MMAL in this case uses a whopping 94% of the CPU used by its non-accelerated counterpart (80.51% vs 85.81% CPU overall). That's not a big difference.

This does not jive with the performance gains mentioned by @ivanich. So are such gains seen with mpv but not ffplay? What gives?

[gy-lehel]

It is not right, and the reason is, that mmal just like most of the OpenGL stack seems to be completely broken in buster+rpi4 combo. I did not check it on RPi3 (my guess is, that there will be no issues on the matching), but on rpi4 even with latest firmware, and compile from scratch one can compile with mmal, but when one tries to use ffmpeg with the h264_mmal decoder all one gets is error messages:

[h264_mmal @ 0x3afc340] Did not get output frame from MMAL. Error while decoding stream #0:0: Unknown error occurred

With default decoder I am able to get ~120fps decoding speed on a fullHD video. I am not able to test viewing as currently I can only use headless to access the pi.

[NoSuck]

... but when one tries to use ffmpeg with the h264_mmal decoder all one gets is error messages:

[h264_mmal @ 0x3afc340] Did not get output frame from MMAL. Error while decoding stream #0:0: Unknown error occurred

Are these error messages also shown with the ffplay command above, using /opt/vc/src/hello_pi/hello_video/test.h264 for $VIDEO? If not, can you please share a command using /opt/vc/src/hello_pi/hello_video/test.h264 that does show them?

[gy-lehel]

ffmpeg -c:v h264_mmal -i /opt/vc/src/hello_pi/hello_video/test.h264 -benchmark -f null -

I can not use ffplay, as I have onlys ssh access to the node.

[gy-lehel]

Well, next step. I did an rpi-update, reboot, and ffmpeg recompile from scratch. This solved the Unknown command error, but you will not believe, mmal assisted decoding tops off at 53 fps, while cpu-only at 107 on the same video...

[gy-lehel]

Following the same procedure in rpi3 + buster combo resulted in dead mmal: mmal: mmal_vc_port_enable: failed to enable port vc.ril.video_decode:out:0(I420): ENOMEM mmal: mmal_port_enable: failed to enable port vc.ril.video_decode:out:0(I420)(0x2328190) (ENOMEM) mmal: mmal_port_disable: port vc.ril.video_decode:out:0(I420)(0x2328190) is not enabled Stream mapping: Stream #0:0 -> #0:0 (h264 (h264_mmal) -> wrapped_avframe (native)) Error while opening decoder for input stream #0:0 : Unknown error occurred

[NoSuck]

Thank you for sharing information. Results here (Pi 4) jive with yours:

■ ffmpeg -c:v h264_mmal -i /opt/vc/src/hello_pi/hello_video/test.h264 -benchmark -f null -
frame=  351 fps= 52 q=-0.0 Lsize=N/A time=00:00:15.04 bitrate=N/A speed=2.24x
video:140kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
bench: utime=3.072s stime=3.235s rtime=6.876s
bench: maxrss=77448kB

■ ffmpeg -c:v h264 -i /opt/vc/src/hello_pi/hello_video/test.h264 -benchmark -f null -
frame=  360 fps= 95 q=-0.0 Lsize=N/A time=00:00:14.40 bitrate=N/A speed= 3.8x
video:143kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
bench: utime=13.264s stime=0.300s rtime=3.794s
bench: maxrss=85024kB

I'm throwing in the towel!

[CounterPillow]

have you considered not using broadcom's abomination of an SBC?

[fitipe]

hi there!

running the latest "raspberry pi os" on a rpi4 with 4gb

I finally could compile ffmpeg with all the mmal and rpi support included. But, when I build the latest version of mpv, it indeed does not find mmal and a bunch of other stuff.

@gy-lehel I'm quite newbie into programming .. how do I use the patches you kindly provided? I've tried to make it a .sh file and run it with sudo, but it's not working ... seems like that's not the proper way of doing it.

thanks a lot!

[oppsig]

Does anyone have a new path that works?