search

davedoesdev / webm-muxer.js

WebM muxer using libwebm and webm-tools compiled to Wasm
http://rawgit.davedoesdev.com/davedoesdev/webm-muxer.js/main/demo.html
MIT License
54 stars 5 forks source link

Support for muxing AV1 #21

Closed AshleyScirra closed 2 years ago

AshleyScirra commented 2 years ago

WebCodecs supports encoding AV1. It would be nice if we could use that to generate WebM files with the AV1 codec. Currently it doesn't work.

It's hard to come by technical details about AV1 in WebM but the best reference I have found is: https://github.com/ietf-wg-cellar/matroska-specification/blob/master/codec/av1.md This is apparently a de-facto standard for WebM too, as browsers support things like WebM files with AV1.

Here's a minimal repro: videoTest.zip

This can successfully encode a VP9 video from a canvas. The only changes are to change the WebM codec_id to V_AV1 and configure WebCodecs to encode AV1. This fails with a few warnings and errors including:

AddGenerocFrame failed. Failed to mux frame: -6 no video, no audio track TypeError: Cannot read properties of undefined (reading 'dataEnd')

davedoesdev commented 2 years ago

Repro'd. Yes it'd be good to support AV, I'll look into it when I get some time.

davedoesdev commented 2 years ago

OK I got this working, just need to tidy it up before I push the changes. Might be a few days.

AshleyScirra commented 2 years ago

Sounds great, thanks for looking in to it!

davedoesdev commented 2 years ago

Should be working now. @AshleyScirra let me know if you try it.

AshleyScirra commented 2 years ago

I tried it out and it works! Thanks, that's really cool.

I was looking at some diffs though and saw some code that looked odd around here: https://github.com/davedoesdev/webm-muxer.js/blob/main/webm-worker.js#L200

First I think there's a typo in the VP9 section - it reads metadata.video.chrome_subsampling whereas I think it was meant to be chroma_subsampling.

Secondly all the AV1 parameters are read off metadata rather than metadata.video as with VP9, so specifying the AV1 params in metadata.video will return undefined for all of the read values and coerce them all to 0.

Despite that it still works, but might be worth a look over that code.

davedoesdev commented 2 years ago

Yes my bad, and it needs a test to vary the params

davedoesdev commented 2 years ago

I've fixed those issues, appreciate the heads up. I also added checking for different private data to the tests.

AshleyScirra commented 2 years ago

All looks good now. Thanks again!