w3c / webcodecs

WebCodecs is a flexible web API for encoding and decoding audio and video.
https://w3c.github.io/webcodecs/
Other
978 stars 136 forks source link

Add ffmpeg-wasm-demuxer sample #549

Closed chcunningham closed 1 year ago

chcunningham commented 2 years ago

Please see a live demo here https://chcunningham-samples.netlify.app/ffmpeg-wasm-demuxer/wasm_demuxer_demo.html

Partial fix for https://github.com/w3c/webcodecs/issues/78

chcunningham commented 2 years ago

@sandersdan, friendly ping :)

dalecurtis commented 1 year ago

Chris left Google and unfortunately won't be able to complete this PR. Chris you mentioned you might want to try to close this out still. Did you want me to leave this open for a bit longer?

KevinBoeing commented 1 year ago

Did somebody already tried to demux a file with a video and audiostream?

Currently I get the following warnings from FFmpeg:

When I try to read a Chunk, FFmpeg gives me the following error:

Invalid data found when processing input

dalecurtis commented 1 year ago

This copy of ffmpeg is built without proprietary codecs, otherwise we wouldn't be able to host/share it. You'll have to tweak the ffmpeg build options if you want other codecs enabled.

dalecurtis commented 1 year ago

Going to close this PR for now since we don't have anyone to finish it at this time. Others are welcome to clean it up and land it if they're so inclined.

chrisn commented 1 year ago

I'm interested to have a look at this if I can make some time.

KevinBoeing commented 1 year ago

Hey, sorry that I still comment under a closed PR but I am currently working on it. I am almost finished with this project, but have two thing I still dont understand / I am struggling with.

  1. Is there an easy way to create a codec string using FFmpeg as required by the WebCodecs decoder configurations (e.g. mp4a.40.2)? I am already aware that I can get important codec information from the AVCodecParameters structure. The prefix (e.g. mp4a) can be determined by the codec_tag property and other important information is in my opinion the codec-profile and the codec-level. However, I'm still not sure how to get a mapping between the FFmpeg AVCodecParameters and the WebCodecs codec string.

  2. In the FFmpegDemuxerBlockingHelper class, the AVIO_BUFFER_SIZE is set to 4,000 bytes. If I increase this value to 20,000 bytes, it works fine with a container that contains only a video stream or an audio stream. If it contains both, I need to set the AVIO_BUFFER_SIZE to the bytes of the container. Otherwise I get a "stream 0/1, offset 0x30 partial file" warning from ffmpeg and when I try to read the next frame (av_read_frame()), I get an error "Invalid data found when processing input". Is it correct that I always have to set the AVIO_BUFFER_SIZE to the size of the container or is there a workaround?