problem playing audio with web audio api

[kboniadi]

Hi, so I have a stream of raw opus encoded audio bytes that I want to decode and play in the browser using web audio api. In the code below, I pass the unsigned char data bytes to the decoder and then convert the decodedPacket to a Float32Array and pass that to the web audo api. But all I get is heavily distorted/static audio playback. I was expecting the decodedPacket size to be 960 (or 960 * 2 for stereo) but the return bytes from the decoder was of size 3840. Could you explain what the decoder is actually returning, and if I even used it correctly in my example code?? For reference I'm using https://github.com/samirkumardas/pcm-player player to play the audio bytes. Thanks

function init() {
  var OpusScript = require("opusscript");
  var sampleRate = 48000;
  // var frameDuration = 20;
  var channels = 2;

  var encoder = new OpusScript(
    sampleRate,
    channels,
    OpusScript.Application.AUDIO
  );

   socket.addEventListener("message", function (event) {
        var decodedPacket = encoder.decode(Buffer.from(event.data));

        var l = decodedPacket.length;
        var outputData = new Float32Array(decodedPacket.length);
        for (let i = 0; i < l; i++) {
          outputData[i] = (decodedPacket[i] - 128) / 128.0;
        }
        console.log(outputData);
        player.feed(outputData);  // splits interleaving array and plays in left and right speaker
   });
}

[abalabahaha]

PCM samples are 16-bit but the return uses Uint8Array/Buffer, so each sample is 2 8-bit elements in the returned array

[Pxlxpenko]

PCM samples are 16-bit but the return uses Uint8Array/Buffer, so each sample is 2 8-bit elements in the returned array

I'm facing the same issue here, don't really understand how to make it work, just use every 2 symbols in a returned sequence to encode 16-bit value?

[abalabahaha]

Oh I just realized the snippet in OP uses Float32Array. Currently this library only supports int16 PCM input/output -- I haven't had the need for float32 yet since this was primarily written for Discord bot audio, though if there is demand I might add that.

Yeah, the returned PCM is little-endian 16-bit PCM in a Node.js Buffer (basically a Uint8Array) -- for a 2-channel stream I think it would look like: [ frame 0 channel 0 lower byte, frame 0 channel 0 upper byte, frame 0 channel 1 lower byte, frame 0 channel 1 upper byte, frame 1 channel 0 lower byte, ... ]

From a quick lookthrough, it looks like PCMPlayer already supports int16-le natively. This works fine for me:

function init() {
  let sampleRate = 48000;
  let channels = 2;

  let player = new PCMPlayer({
    encoding: "16bitInt",
    channels: channels,
    sampleRate: sampleRate,
    flushingTime: 1000,
  });

  let encoder = new OpusScript(sampleRate, channels, OpusScript.Application.AUDIO);

  let socket = new WebSocket("ws://127.0.0.1:8081");
  socket.binaryType = "arraybuffer";
  socket.addEventListener("message", function (event) {
    let data = new Uint8Array(event.data);
    let decodedPacket = encoder.decode(data);
    player.feed(decodedPacket);
  });
}
window.addEventListener("load", init);

Server for reference (there's other sources of Opus packets than my Discord library but I was lazy):

const WebSocket = require("ws");
const fs = require("fs");

function getOpusStream() {
  const OggOpusTransformer = require("eris/lib/voice/streams/OggOpusTransformer");
  return fs.createReadStream("./file.opus")
    .pipe(new OggOpusTransformer({ objectMode: true }));
}

let wss = new WebSocket.Server({ port: 8081 });
wss.on("connection", async (client) => {
  console.log("Connected, sending stream");

  // Node.js object-mode readable stream (read() -> 1 Opus packet Buffer)
  let stream = getOpusStream();
  let msPerPacket = 20;
  let msPerBatch = 1000;
  let interval = setInterval(() => {
    for(let i = 0; i < (msPerBatch / msPerPacket); i++) {
      let packet = stream.read();
      if(!packet) return;
      if(client.readyState !== WebSocket.OPEN) {
        console.log("Disconnected");
        return clearInterval(interval);
      }
      client.send(packet);
    }
  }, msPerBatch);
});
console.log("Listening");