search

bvibber / ogv.js

JavaScript media player using Ogg/Vorbis/Theora/Opus/WebM libs compiled with Emscripten
https://brooke.vibber.net/misc/ogv.js/demo/
Other
1.2k stars 101 forks source link

How can I use ogv just for decoding? #533

Open jfrancos opened 5 years ago

jfrancos commented 5 years ago

For Brave/Firefox I use WebAudio's decodeAudioData, which returns an AudioBuffer of the decoded PCM audio data. This doesn't work in Safari.

Is there a way I can use ogv just for decoding, in place of decodeAudioData?

Thanks for your hard work.

bvibber commented 4 years ago

This would be possible to rig up using the codec modules directly, though not yet a convenient interface for it.

Roughly, you need to pass data into a demuxer module's receiveInput method, then loop over it with process, dequeuing any packets along the way and passing them into the decoder module. Take the output data, concatenate it all into a single ArrayBuffer and resolve the promise.

Devil's always in the details though. :)

guest271314 commented 4 years ago

What is input and expected output?

guest271314 commented 4 years ago

Do not have access to Safari. Testing decodeAudioData at Epiphany 3.28.5 an AudioBuffer is logged when the audio codec is Ogg or A_PCM/FLOAT/IEEE in a WebM container or MP2/MP3 in a MP3. OPUS in a WebM container outputs null. Conversion to WAV can be achieved using this version https://cdn.jsdelivr.net/npm/audiobuffer-to-wav@1.0.0/index.js of https://github.com/mattdiamond/Recorderjs at https://plnkr.co/edit/HMqf5CrAXhOHtlzOadI8?p=info.

jfrancos commented 4 years ago

Actually I figured out how to use emscripten to be able to use vorbis-tools' oggdec directly:

https://github.com/jfrancos/oggdec-wasm

guest271314 commented 4 years ago

@jfrancos Are you able to verify which file types and codecs Safari is able to decode with decodeAudioData() re https://github.com/brion/ogv.js/issues/533#issuecomment-565857607 and https://github.com/mdn/webaudio-examples/issues/5?

rgov commented 3 years ago

I would love to be able to do this as well. My use case is that I have a stream of "raw" Theora packets coming from a camera, without an OGV container. I want to be able to feed in the header packets, an intra frame, and a sequence of inter frames and get out the final image buffer. Is this possible at all? The codec-bench.html file looks like it might be close.

bvibber commented 3 years ago

Yes, you can call the decoder directly if you like. No stable api or documentation or support.

rgov commented 3 years ago

Ok thanks. I put together an example of using ogv.js to decode raw Theora packets into video frames: https://github.com/rgov/js-theora-decoder (Demo)

@brion One thing I'm curious about is why I need to call processHeader() for intra frames, instead of processFrame().