godotengine / godot-proposals

Godot Improvement Proposals (GIPs)
MIT License
1.16k stars 97 forks source link

Add MP4/WebM/AV1 import through an FFMPEG conversion wrapper #7062

Open TechnoPorg opened 1 year ago

TechnoPorg commented 1 year ago

Describe the project you are working on

An educational animation project that plays video content from a file as part of a visual example.

Describe the problem or limitation you are having in your project

Because the codecs used in the MP4 format cannot be used in Godot due to licensing issues, Ogg Theora is the only format supported for video playback in the engine. This is already well documented (https://docs.godotengine.org/en/stable/tutorials/animation/playing_videos.html), and there is now the possibility for a third-party plugin implementing MP4 to be created, but no one has done so yet. Because most video programs work with the MP4 format, this makes it cumbersome to use most video files in Godot, as they first need to be converted to Ogg Theora.

Describe the feature / enhancement and how it helps to overcome the problem or limitation

It would make it considerably simpler to use MP4 files in Godot by using an import process similar to that of the FBX format. MP4 files would not be directly used by the engine, but rather be converted behind the scenes using FFMPEG into an Ogg Theora file, and that is what would be imported and used for video playback. I believe this would not lead to any MP4 patent issues, as this would simply streamline the existing conversion workflow by making the engine convert the MP4 file, rather than the user.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

In the editor settings, users wishing to use this feature would provide a path to an FFMPEG binary, which could then be called by the editor to convert the video file. Using FFMPEG to convert MP4 files is already documented in the aforementioned "Playing Videos" section of the documentation, so that would likely be the command used. Extra quality settings could potentially be added to provide finer-grained control.

If this enhancement will not be used often, can it be worked around with a few lines of script?

It could be worked around with an editor plugin, either one implementing MP4 playback directly, or one performing this "conversion import".

Is there a reason why this should be core and not an add-on in the asset library?

An add-on needs to be installed for every project, and it would be nice if the most ubiquitous video format just worked out of the box in every Godot project. This would also not be a large change to the engine, as all the heavy lifting is done by FFMPEG, and this would just be a wrapper.

fire commented 1 year ago

👍 I like this.

This proposal suggests using FFmpeg to auto-convert MP4 to Ogg Theora during import, like FBX files.

Provide FFmpeg binary path in editor settings

Implementing as an editor plugin is an option

In summary:

References

This comment was ai assisted.

Calinou commented 1 year ago

I feel this proposal glosses over one of the largest advantages of using H.264 over Theora: hardware-acclerated decoding. This is widely supported with H.264 nowadays – you'd be hard-pressed to find hardware that doesn't support it. As a result, I'm not sure if many people would use this FFmpeg conversion approach as you'd lose on both quality and performance compared to using H.264 directly (not to mention all the downsides of double-lossy encoding).

While patents make direct inclusion in Godot problematic, we can probably find a way to use the system decoders, which would allow for hardware acceleration and make the patent issue "not our problem" so to speak. It's difficult to do for all platforms, but the effort would be worth it.

fire commented 1 year ago

I have prs for direct windows api calls, but I wasn't able to complete.

https://github.com/V-Sekai/godot/compare/3.5-wmf

https://github.com/V-Sekai/godot/commits/wmf-4

frankhuurman commented 2 months ago

I was actually looking to build some sort of customized video editor in Godot but noticed the lack of video file support. Is there any update on this since the issue was opened? Also would love the ffmpeg conversion on import.

fire commented 2 months ago

Are you familiar with the 2+ projects that use ffmpeg? I don’t have the links on hand.

Eon-Able commented 2 months ago

You guys are looking probably for either: https://github.com/VoylinsGamedevJourney/gde_gozen or this one: https://github.com/EIRTeam/EIRTeam.FFmpeg

Both integrate support for video codec through ffmpeg. I still firmly believe for a wide number of games, that rely on animations etc. Godot needs a better format than ogg theora in it's core.

But in the meantime that should help you. As for the proposal on top. I don't think it makes much sense. As you can use ffmpeg to encode your videos by yourself. The workaround is minimal and something any game dev working with video files knows.