anthumchris / fetch-stream-audio

Low Latency web audio playback examples for decoding audio streams in chunks with Fetch & Streams APIs
https://fetch-stream-audio.anthum.com
MIT License
378 stars 23 forks source link

Add Opus decoding support #4

Closed anthumchris closed 4 years ago

anthumchris commented 6 years ago

Chunk-based Opus file decoding should be supported. Emscripten can be used to build WebAssembly modules for decoding the data. The opusfile decoder library could be used if it supports chunk-based byte array decoding. Initial research shows that a complete file may be needed for OggOpusFile objects. Otherwise, libogg may be needed to interface directly to parse/assemble the chunks into decodable packets.

The preference is to use WebAssembly for all decoding functions and limit use of JS to parse Ogg file structure. mohayonao/libogg.js could be used if JS is needed.

anthumchris commented 6 years ago

local chunk-based decoding tests succeed using op_open_callbacks. Will clean up code and push

haywirez commented 6 years ago

👍 Let me know if you need help with this. I have zero experience with decoders so would need some patient pointers from time to time, but also trying to get a HLS stream containing .opus/.ogg files to work...

anthumchris commented 6 years ago

@haywirez I dropped this a few months ago and would be happy to pick up again. Also, decoding would be the easy part, but seeking/queuing would be more difficult.

I got as far as branch https://github.com/AnthumChris/fetch-stream-audio/tree/opus-stream-decoder. It's a sloppy branch that is undocumented and not ready for handoff, and it's been too long for me to remember exactly where it left off. I remember getting as far as command-line tests that pass an .opus file to WASM via NodeJS and then writing the decoded PCM data to a wave file to compare. I think my next steps were to handle deinterlacing in C and return two arrays (left/right channels) that could be consumed by a WebWorker.

I'll be busy for a couple weeks and apologize I can't clean this up and give you a proper handoff before then. Now that someone else is interested, it's on my to-do list.

lavvy commented 6 years ago

Am interested too

anthumchris commented 6 years ago

@haywirez @lavvy OpusStreamDecoder is now ready and available as a standalone module below. I'll integrate into this demo repo as a submodule when I have time.

https://github.com/anthumchris/opus-stream-decoder

If you could, please test the build and file any issues you notice in opus-stream-decoder