duvallj
commented
1 month ago Due to limitations in the C++ APIs, onData can only accept packets containing exactly 10ms of audio at the given sampleRate you provide.
Here's an example for what I've done to chunk the data properly:
class AudioSource {
private leftoverSamples: Int16Array;
private numLeftoverSamples: number;
private numberOfSamplesPerFrame: number;
constructor(
private audioSource: nonstandard.RTCAudioSource,
public readonly samplerate: number,
) {
// The number of frames wanted by node-webrtc when exporting audio data,
// exactly 10ms at the given sample rate. This is used internally by
// libwebrtc and cannot be changed.
// (samples / 10ms) = (samples / s) / (1000ms / 1s) * 10
this.numberOfSamplesPerFrame = samplerate / 100;
this.leftoverSamples = new Int16Array(this.numberOfSamplesPerFrame);
this.numLeftoverSamples = 0;
}
public presentSamples(samples: Int16Array) {
// Procedure:
// 1. Fill up leftoverFrames with enough data to make a complete frame
// 2. Send out as many complete frames as possible
// 3. Put all remaining data into leftoverFrames
let chunkStart = 0;
while (chunkStart < samples.length) {
const wantedNumberOfSamples = this.numberOfSamplesPerFrame - this.numLeftoverSamples;
const remainingSamples = samples.length - chunkStart;
if (remainingSamples < wantedNumberOfSamples) {
this.leftoverSamples.set(samples.slice(chunkStart));
this.numLeftoverSamples = remainingSamples;
break;
}
let chunk = samples.slice(chunkStart, chunkStart + wantedNumberOfSamples);
if (this.numLeftoverSamples) {
this.leftoverSamples.set(chunk, this.numLeftoverSamples);
chunk = this.leftoverSamples;
this.numLeftoverSamples = 0;
}
this.audioSource.onData({
samples: chunk,
numberOfFrames: this.numberOfSamplesPerFrame,
sampleRate: this.samplerate,
});
chunkStart += wantedNumberOfSamples;
}
}
}Sorry it's so complicated...
royibernthal
I'm trying to have my nodejs backend act as a peer that streams pcm audio to the browser. (let's assume it's the only way to achieve what I need)
I'm creating an RTCAudioSource, and adding to it an array of chunked buffers - PCM 24k sample rate signed 16-bit little-endian.
RTCAudioSource/onData only accepts an Int16Array in samples, so I tried converting the buffers:
source.onData() throws the following error, with x being anywhere from 17 to 2770 depending on the chunk size:
It's worth noting that the PCM buffers play fine when writing them to a wav file or when emitting and playing them in the browser via web sockets.