search

UltraStar-Deluxe / Play

Free and open source singing game with song editor for desktop, mobile, and smart TV
https://ultrastar-play.com
MIT License
389 stars 72 forks source link

Problem playing vp8/webm files on Android. Which video format for cross-platform support? #280

Closed achimmihca closed 2 years ago

achimmihca commented 2 years ago

I expected Unity to support vp8 video files on Unity. Unity docs say:

When cross-platform support is a high priority, [VP8](https://en.wikipedia.org/wiki/VP8) is a good choice.
It’s widely supported and has a comprehensive feature set, [...]

Still, I see errors in Android Logcat when trying to use a vp8 file:

2022.03.20 08:59:43.439 1389 1487 Verbose NuMediaExtractor NuMediaExtractor constructor
2022.03.20 08:59:43.440 1389 1487 Error NdkMediaExtractor can't create http service
2022.03.20 08:59:43.441 1389 1487 Warn Unity AndroidVideoMedia: Error opening extractor: -10002
2022.03.20 08:59:43.441 1389 1487 Verbose NuMediaExtractor NuMediaExtractor destructor
2022.03.20 08:59:43.447 1389 1429 Error Unity VideoPlayer cannot play url : file:///storage/emulated/0/Android/data/com.Karaoke.UltraStarPlay/files/Songs/d - d/d - d.webm
2022.03.20 08:59:43.447 1389 1429 Error Unity Cannot read file.

I tried this with both vp8 and webm file extension. Makes no difference.

On the other hand, mp4 files works on Android. Sadly, mp4 does not work on Linux but vp8 does (see #246).

Question: is there a single video file format that works out of the box at runtime with Unity's VideoPlayer.url on Windows, MacOS, Linux, Android and iOS?

basisbit commented 2 years ago

I'd suggest asking this in the Unity forum

achimmihca commented 2 years ago

I'd suggest asking this in the Unity forum

See https://forum.unity.com/threads/videoplayer-url-issue-with-vp8-webm-on-android-androidvideomedia-error-opening-extractor-10002.1255434/

achimmihca commented 2 years ago

I found a recent roadmap of Unity for audio/video: https://unity.com/roadmap/unity-platform/audio-video A recently (08.02.2021) planned feature is "Runtime Decoding":

The Unity media framework exposes all the building blocks needed to make various workflows such as low-level decoding, encoding, playback, recording, etc. This first implementation phase of the framework will let users leverage raw audio/video decoding for situations where custom content needs to be ingested at runtime (e.g.: custom feeds from devices, data from proprietary file formats, etc). We’re aiming to expose availability of the H.264 video decoder and AAC audio decoder on all platforms. Developers would also be able to add their own implementation of these decoders if they wish to go beyond the capabilities of the stock implementations.

This should enable mp4 support on all platforms. My only doubt is that Unity tends to need decades to go from "planned" to "released" for a feature... but still nice!

bohning commented 2 years ago

So to be precise, this should enable support for mp4 containers with H.264 encoded video and m4a containers with AAC encoded audio, I assume?

H.264 encoded video can be in many different file containers and AAC encoded audio can be in many different file containers.

basisbit commented 2 years ago

Container format doesn't really matter - support for those can be implemented in managed c# code if necessary and ffmpeg based scripts can be provided / documented which convert the files into the desired container format losslessly. As soon as hardware-accelerated x264 decoding is supported, we should be able to get an easy to use best-practice zip/tar package format working, which should make sharing/embedding (properly licensed) content easier.