godotengine / godot

Godot Engine – Multi-platform 2D and 3D game engine
https://godotengine.org
MIT License
91.22k stars 21.22k forks source link

Use hardware decoder on various platforms to decode videos such as h.264 #12362

Closed kurikaesu closed 4 years ago

kurikaesu commented 7 years ago

This is a proposal to write for various platforms the ability to use the underlying hardware video decoder to decode h.264 files to compliment the software theora decoder.

On low powered devices such as tablets, the theora decoder does not always keep up to be able to render a 24-30fps video at fairly high resolutions.

Is there any interest in this by anybody else in the community? My team will be implementing this regardless as it is a requirement for the product we're trying to build.

NathanWarden commented 7 years ago

I'm definitely not the person to say you should or shouldn't do it, but this sounds like a welcomed addition. I mostly work on client projects and they occasionally want things like video, so I'm all for it :)

eon-s commented 7 years ago

I guess it was not possible due to patents, not sure the current status but even a module/native will be nice to have.

reduz commented 7 years ago

This should be much better added with gdnative eventually

On Oct 23, 2017 9:07 PM, "eon-s" notifications@github.com wrote:

I guess it was not possible due to patents, not sure the current status but even a module/native will be nice to have.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/godotengine/godot/issues/12362#issuecomment-338831778, or mute the thread https://github.com/notifications/unsubscribe-auth/AF-Z22b4Qrd47PeWk7ZSxNp5PggxjpI9ks5svSo9gaJpZM4QDjKn .

kurikaesu commented 7 years ago

Patent wise, I'm not a lawyer but I believe that using the built in decoders of a platform such as iOS, Android etc, will not encumber your software with the related h.264 patents as the respective companies (Apple, Motorolla, Microsoft etc) have already paid the licensing fees for the decoder and you receive a license agreement from those vendors relating to h.264.

If the proposal was to write a software decoder into Godot, then patents would definitely be an issue.

kurikaesu commented 6 years ago

My coworkers and I at my company have started implementing this for Windows to start. You can see our progress at https://github.com/DisplaySweet/godot/commits/plugin/wmf-wip

We will be moving the code out of our Godot repository into its own repository that can be included into Godot at will by anyone.

kurikaesu commented 6 years ago

My co-worker Matt has completed the video decoding and playback for H.264 in Windows which performs quite well.

I'm currently investigating how to use the video card hardware decoder when present in order to increase the performance but most documentation on windows refers to using the DirectX 11 APIs to use DXVA. Not too sure if there is a better way of decoding other than copying the texture out of the graphics card and back into the graphics card when we set the texture data.

eon-s commented 6 years ago

@kurikaesu I would recommend you to go to IRC, godotengine (general) or godotengine-devel (engine dev) channels on freenode to contact devs, they may be able to help better there.

aaronfranke commented 5 years ago

Has anyone made any progress turning this into a GDNative plugin?

Calinou commented 4 years ago

Feature and improvement proposals for the Godot Engine are now being discussed and reviewed in a dedicated Godot Improvement Proposals (GIP) (godotengine/godot-proposals) issue tracker. The GIP tracker has a detailed issue template designed so that proposals include all the relevant information to start a productive discussion and help the community assess the validity of the proposal for the engine.

The main (godotengine/godot) tracker is now solely dedicated to bug reports and Pull Requests, enabling contributors to have a better focus on bug fixing work. Therefore, we are now closing all older feature proposals on the main issue tracker.

If you are interested in this feature proposal, please open a new proposal on the GIP tracker following the given issue template (after checking that it doesn't exist already). Be sure to reference this closed issue if it includes any relevant discussion (which you are also encouraged to summarize in the new proposal). Thanks in advance!

fire commented 2 years ago

@kurikaesu

I attempted to port over the wmf code to the latest Godot Engine 3 and master.

https://github.com/V-Sekai/godot/tree/3.5-wmf and https://github.com/V-Sekai/godot/tree/wmf were made.

I was able to use the 3.5 wmf for both audio and video, but Godot Engine 4 could only play video.