search

love2d / love

LÖVE is an awesome 2D game framework for Lua.
https://love2d.org
Other
4.7k stars 384 forks source link

AV1 video playback #1625

Open slime73 opened 3 years ago

slime73 commented 3 years ago

Ogg Theora (which love.video currently supports) isn't very good as a format in general, and the backend code in love is hard to maintain and has bugs. The main reason Theora was used in the first place is at the time all the alternatives had other major downsides that made them less viable.

Now AV1 exists, and while decoding it on the CPU is expensive, dav1d seems like it should be fast enough while also having a simple API, a permissive license, a relatively small library size, and no patent issues. It doesn't provide cmake build setup (currently), so we'd have to create that.

Ideally we'd completely remove Ogg Theora support in favour of AV1.

MikuAuahDark commented 3 years ago

Don't worry about CMake build setup. I've created one here https://github.com/MikuAuahDark/dav1d-cmake

It's not the latest version but should be easy to modify to match upstream.

Personally I'm against AV1 because none of the libraries seem provide promising performance without ASM. Not sure if we should add "NASM" as another dependency to LOVE just for building dav1d. Also encoding video to AV1 takes ages even with high-end computer, which makes people more less interested on it.

Regarding Theora, maybe some bugs were fixed when we update Theora to 1.2.0-alpha1 (like in Android) or to upstream. This needs more testing though.

Another alternative is to use OS-specific API for video decoding. This will require us to write lots of OS-specific codes which is not ideal, but we can support many video formats this way. Windows has MediaFoundation, Linux has FFmpeg or GStreamer, Android has Media API (but the NDK interface only exists when targetting Lollipop).

slime73 commented 3 years ago

none of the libraries seem provide promising performance without ASM. Not sure if we should add "NASM" as another dependency to LOVE just for building dav1d.

All we have to do is only build AV1 support on architectures dav1d natively supports (x86/x64/armv7/arm64) and there should be no extra dependencies, right?

encoding video to AV1 takes ages even with high-end computer

That is a bit of an issue, but encoding is something people only have to do once for a video, and tools support for AV1 is being adopted pretty much everywhere. Unlike Theora AV1 is on track to become a real standard format.

MikuAuahDark commented 3 years ago

All we have to do is only build AV1 support on architectures dav1d natively supports (x86/x64/armv7/arm64) and there should be no extra dependencies, right?

Building for x86/x64 requires "NASM" (external program) to build the assembly-optimized program, which translates pretty much to "mandatory" given the state of current AV1 CPU decoding requirements.

AV1 is on track to become a real standard format.

Yet we still need to add support for video containers like MP4 or MKV (and demuxing it). Unlike AV1, Theora commonly stored in Ogg container and can be demuxed with libogg, saving the hassle of additional demuxer library.

bartbes commented 3 years ago

Yet we still need to add support for video containers like MP4 or MKV (and demuxing it). Unlike AV1, Theora commonly stored in Ogg container and can be demuxed with libogg, saving the hassle of additional demuxer library.

I'm a bit fuzzy on the details, but when I was implementing vp8/webm support a while back, I did also implement a webm (/mkv!) demuxer. A result of that experiment was separating out demuxing and decoding in the existing codebase. At one point I was displaying vp8 from an ogg container and theora from an mkv container, so that is all already supported.. in theory.

MikuAuahDark commented 3 years ago

Another alternative to dav1d for AV1 video decoding is libgav1. It provides CMake build scripts and doesn't need NASM to build on x86(-64) platforms, but this means I have to write Android.mk for Android (which is probably not easy, I haven't looked into detail of their CMake scripts).

dvn0 commented 2 years ago

What about using libmpv?

MikuAuahDark commented 2 years ago

We can't use libmpv because it's LGPL, which means we can't ship it for iOS. Same reason why we didn't use FFmpeg in the first place.

ericoporto commented 10 months ago

@MikuAuahDark , hey, about libgav1, I see it pull abseil as a dependency. While Theora has it's flaws it's a very small, easy to build dependency, have you played around libgav1? I like it uses CMake and has no nasm requirement, and it uses the Apache License, which looks ok for iOS.

MikuAuahDark commented 9 months ago

libgav1 sounds promising, but I don't think this is being in our high priority at the moment.

ericoporto commented 9 months ago

Ah ok, so I built libdav1d.dll here to have an idea (used msys2), and it's around 3MB and go to 2MB after using strip. I haven't tried to build libgav1 yet. With love being 3.5MB it's quite hard to find something as small I imagine.