Example of broadcasting audio data over WebSockets?

[ablydevin]

I'm working on a simple audio broadcast app that intends to send audio from one browsers microphone to another browser for playback via WebSocket (using Ably). I realize Websockets might not be intended for this, but I'm enjoying the challenge. I've been trying to use the two different examples you have in this repo to cobble it together.

On the "publisher" side I have an AudioWorkletProcessor that sends microphone data to a Web Worker (using ringbuf) which publishes the data to an open WebSocket.

On the "subscriber" side I have a WebSocket listener that takes audio data and passes it to a Processor (via ringbuf).

The audio data appears to be being passed through the socket but it's not playing on the subscriber side. I'm a newb to Web Audio APIs and audio processing so I'm kind of stumbling my way along. It's possible that I'm not transforming the raw audio on the publishing side in a way that can be passed well through the socket. It's also possible that I need to do some kind of transformation on the subscriber side to get the data playable. I suppose it is also possible that the WebSocket is changing the data 🤷‍♂️

Anyway, I'm taking a shot in the dark that you'd be down to help sort this out with me.

Thanks!

[ablydevin]

Quick follow-up on some progress. I figured out that the subscriber gets data from the WebSocket as an arraybuffer and I need to convert it to Float32Array. I added a stackoverflow-sourced function to convert and that does result in sound on the subscriber side - but it's noise, not the audio I want. So one step forward - but more to go.

[padenot]

WebSocket isn't the right tool for this if it's supposed to be real-time, but anyway, it can be done I suppose if you don't care about latency at all. It's also absolutely wasteful to not process/compress the data.

Anyway, what you need to do is to interleave the audio data from the mic (if it's not mono), write to the ring buffer, read from the worker, write in the socket, read from the socket on the other side, turns the ArrayBuffer into a Float32Array (this should just be new Float32Array(arraybuffer), de-interlerleave the data, and play it out using another ring buffer.

It's critical to buffer quite a bit, before playing it on the read side because otherwise the audio buffers will be intermixed with silence and that can sound like noise. But checking that you read what you wrote on the other side is a good first step.

In general, I want to stress that this is not how voice communication works well. Also audio on the web is always ever float 32 bits.

I'll close this because this is not an issue with the library, and sending audio shouldn't be done using websockets and PCM, but good luck!