search

gfx-rs / wgpu

A cross-platform, safe, pure-Rust graphics API.
https://wgpu.rs
Apache License 2.0
12.79k stars 938 forks source link

Support video encoding and decoding APIs #2330

Open zmerp opened 2 years ago

zmerp commented 2 years ago

Vulkan Video extensions were announced earlier this year and GPU vendors are working on the support. At the time of writing, this/this is the state of the art.

More recently it was announced the Direct3D 12 Video API. Encoder support landed this month.

It would be valuable for wgpu to provide a backend agnostic API for video encoding/decoding, even within a separate crate.

dsseng commented 2 years ago

Vulkan video is still in the development. Support pending for AMD/Intel, so first these can be used: Linux - VA-API (can be made to work over VDPAU on NVIDIA) macOS - AVFoundation (does MoltenVK already support Vulkan video extensions?) Windows - DX12 Video API (probably DXVA on DX11?)

Probably waiting for at least a couple months will be better, since RADV/ANV/MoltenVK/Windows drivers need to mainline and stabilize extensions, then something could be made as an example on top of ash and then integrated into wgpu. It would be a bit hard to implement video on backends other than Vulkan, since it would be necessary to import frames as DMA-BUFs (or similar objects named differently on a platform) from another library to maintain zero-copy GPU memory ops (similar to #1733, #2320 as well).