search

raspberry-vanilla / android_local_manifest

106 stars 48 forks source link

Options for better AV1 decoding #10

Closed nename0 closed 1 year ago

nename0 commented 1 year ago

Currently (at least for me) watching AV1 1080p on YouTube is just impossible.

I read that ffmpeg 6.0 now has better support for AV1. Mostly it seems to be about GPUs, that of course would not work on the raspberry pi, but this dav1d library in ffmpeg is also mentioned:

https://github.com/raspberry-vanilla/android_external_ffmpeg/blob/e7fb33135857398e6bada301cc6d311e5c35ef66/configure#L230

which seems to be also optimized for ARM and ARM Neon (https://code.videolan.org/videolan/dav1d#roadmap)

Also it talks about Android MediaCodec and NDKMediaCodes not sure if this helps us...

As far as I can tell we already have all the necessary media codec background stuff to make ffmpeg/libavcodec be the video decoder on android (https://github.com/raspberry-vanilla/android_external_ffmpeg_codec2/blob/android-13.0/media_codecs_ffmpeg_c2.xml). So adding this should not be super huge?

Do you think this would be a viable option for better decoding of AV1 ?

KonstaT commented 1 year ago

Raspberry Pi hw codecs support is not mainlined in FFmpeg so updating to 6.0 is not an option until jc-kynesim updates his fork (https://github.com/jc-kynesim/rpi-ffmpeg/).

Configuring FFmpeg to use libdav1d is probably not much of an issue. Compiling libdav1d for Android could be a bit more complicated as it uses meson build system (Mesa and libcamera also use meson to build which can be used for reference). ffmpeg_c2 plugin also has no support AV1 and it probably requires a bit more than just advertising it in media_codecs.xml.

As it's still a software decoder I have no idea how well it would perform in any case.

YouTube seems to use H.264/VP9/AV1 depending on the video and platform (app/browser/etc). Some third party YouTube apps e.g. SkyTube have option to use legacy formats so you can use this to test H.264 hardware decoding as well.

KonstaT commented 1 year ago

I've updated FFmpeg to 6.0 and implemented AV1 software video decoding using dav1d in android-13.0 and android-12.1 branches.

It certainly performs better than Android's c2.android.av1.decoder software decoder which basically didn't manage to play anything. Using FFmpeg and dav1d, AV1 plays fine up to 720p even with higher bitrate. 1080p also plays but only on videos with very low bitrate.

Good source for AV1 test clips. https://test-videos.co.uk/bigbuckbunny/mp4-av1

nename0 commented 1 year ago

Nice 👍