search

hexops / mach

zig game engine & graphics toolkit
https://machengine.org
Other
3.02k stars 145 forks source link

add and bless AV1 video encoding/decoding package #998

Open slimsag opened 10 months ago

slimsag commented 10 months ago

Similar to how we have mach-flac and mach-opus, which are going to be our blessed audio formats, we should have bindings for encoding and decoding AV1 video as our blessed video format.

mateli commented 9 months ago

Do we support hardware encode/decode? Zero-copy when done on the GPU (dma-buf)? Other interesting compression news. Perhaps AI can be used to achieve a much higher compression ratio than AV1 and other traditional codecs. https://hexus.net/tech/news/software/146086-nvidia-uses-ai-reduce-video-conferencing-data-use-10x/

When generating video from tools like blender we have the option of creating some compact vector based format that do not need further compression. Simply by not having pre-rendered video and instead rendering it while playing.

It could also be possible to use AI to deconstruct rendered video into a compact vector based format.

slimsag commented 8 months ago

Probably we'll go with https://gitlab.com/AOMediaCodec/SVT-AV1 - it looks quite good, C99, has an encoder and decoder, seems fast (built by Intel and Netflix), etc.

Other options either don't offer an encoder (we need one), or would require a C++ and/or Rust compiler.

Do we support hardware encode/decode? Zero-copy when done on the GPU (dma-buf)?

Most libraries for this I think are software-only currently, AV1 hardware support is not widespread yet. There exist extensions for Vulkan, Metal, and d3d12 that we can leverage long-term, but this will need to be done in sysgpu as an extension/optional feature until hardware support is broad enough.