search

Vanilagy / webm-muxer

WebM multiplexer in pure TypeScript with support for WebCodecs API, video & audio.
https://vanilagy.github.io/webm-muxer/demo
MIT License
197 stars 12 forks source link

Support transparency / alpha by encoding stream #19

Closed akre54 closed 1 year ago

akre54 commented 1 year ago

Hi there - I'm trying to use this library to encode a VideoFrame sequence with alpha to the container (ref #9). I see that it should be possible to write an alpha channel to VP8 because WebMWriter does it, using BlockAdditions / BlockAdditional. Would it be feasible to add to this library?

Thanks for the great work!

Vanilagy commented 1 year ago

This library already has alpha support! Look at the muxer config definition. The problem is that VideoEncoder, at least in Chromium, doesn't support alpha video encoding. Or was that not what you were asking?

I have yet to test this library's alpha capability, even though I added it, since VideoEncoder is holding me back.

akre54 commented 1 year ago

I guess I'm not familiar enough with what parts live where, but yes that does seem to be what I'm asking. It looks like WebMWriter doesn't use VideoEncoder but is able to get alpha channel output. Is there a hackaround for this, or would it be possible to shim the webcodecs api? (say, with an emscripten port of vp9) or is this a pipe dream? Thanks for your help here

akre54 commented 1 year ago

Could I, say, pass a Uint8Array to muxer.addVideoChunk, or is VideoEncoder doing something very complicated here?

Vanilagy commented 1 year ago

I see! Yes, if you encode the frames yourself, you could use a WASM port of VP9 to encode with an alpha channel. This library has the .addVideoChunkRaw method for this usecase! You simply pass it a Uint8Array and some metadata.

akre54 commented 1 year ago

Oh amazing. Let me give that a shot

Vanilagy commented 1 year ago

Any updates? Can I close this issue, or do you still need help with something?

akre54 commented 1 year ago

Nope! I ended up using WebM Writer directly, but hoping they'll add alpha support to the codecs soon.

Thanks for your help

galipmedia commented 1 year ago

I am trying to do the same thing. In webm-muxer it didn't fail but the file didn't work. When I tried WebM Writer it rendered my pixi text animations really terribly with a lot of artifacts. I also can't wait for webcodecs to support alpha :(