benz2012
commented
1 year ago Some part of me thinks this is outside of the scope of this project, as it is meant to be a port of FFMPEG into a Javascript environment -- not an "all-purpose best-built video encoder for the browser". I think your idea could lead to the potential for a larger library that wraps multiple solutions like ffmpeg.wasm, VideoEncoder API, MediaRecorder API, etc
However, I generally like the idea and where your head is at
I just heared that chrome added an API to support raw native h264 encode and decode.
https://caniuse.com/mdn-api_videoencoder
https://caniuse.com/mdn-api_videodecoder
https://caniuse.com/mdn-api_audioencoder
https://caniuse.com/mdn-api_audiodecoder
https://github.com/vjeux/mp4-h264-re-encode
I'm not too familiar with emscripten, but it looks like it is possible to share memory between javascript and c. So my idea is to implement those decoder/encoder apis into ffmpeg as sort of an hardware acceleration.
This could resolve some of the "slow" tickets like: https://github.com/ffmpegwasm/ffmpeg.wasm/issues/326 https://github.com/ffmpegwasm/ffmpeg.wasm/issues/206 It could also make it viable to run ffmpeg single core and avoid the isolation requirements: https://github.com/ffmpegwasm/ffmpeg.wasm/issues/285
We'd still have the c implementation of said codecs as a fallback, the demuxers and muxers and we'd still have all the filters of ffmpeg. So I think it is still worth it to use ffmpeg.