Update README.md

[guest271314]

Add use case: Implement infinite audio streams, Web radio stations.

Reference: Issue 1161429: FLAC and Opus Audio Streams Stop Playing https://bugs.chromium.org/p/chromium/issues/detail?id=1161429

[padenot]

At the very list this needs to be in the first list, we're not recording from a worklet here.

This is also generally a very bad idea. This use-case is much better served by just using an HTMLAudioElement. The fact that a browser engine has a bug doesn't change this, and we shouldn't encourage authors to do this, as it has implications in terms of resilience against load and power comsumption, amongts other things.

[guest271314]

I am having a challenging time trying to reproduce an infinite stream to test the claim that Opus and FLAC streams arbitrarily stop while MP3 streams do not.

I began a test yesterday that I expected to last for at least 48 hours. However it only lasted 2 hours.

<?php 

 if (isset($_GET["start"])) {
    header('Vary: Origin');
    header("Access-Control-Allow-Origin: http://localhost:8000/");
    header("Access-Control-Allow-Methods: GET, OPTIONS, HEAD");
    header("Access-Control-Allow-Headers: Content-Type, Access-Control-Allow-Headers");    
    header("Content-Type: application/ogg");
    header("X-Powered-By:");
    echo passthru("ffmpeg -i https://streaming.live365.com/a48993 -f wav - | opusenc - -");
    exit();
  }

Encoding complete                           
-----------------------------------------------------
       Encoded: 2 hours 32.6 seconds
       Runtime: 2 hours 3 seconds
                (1.004x realtime)
         Wrote: 70112113 bytes, 361630 packets, 7235 pages
       Bitrate: 76.8789kbit/s (without overhead)
 Instant rates: 1.2kbit/s to 196kbit/s
                (3 to 490 bytes per packet)
      Overhead: 0.867% (container+metadata)

I have streamed using HTMLMediaElement.srcObject = MediaStream from getUserMedia() for over 24 hours, however not every user is comfortable creating virtual devices that point to sources other than microphone at the command line.

Once I go through every line of code in this repository I will try the infinite stream approach using the techniques disclosed herein (overwriting) substituted for just growing memory https://github.com/guest271314/webtransport/blob/main/webTransportAudioWorkletWebAssemblyMemoryGrow.js.

If the ring buffer truly does just overwrite the indexes of a single pre-allocated memory block that have already been written what is the effective difference between HTMLAudioElement and AudioWorklet for the use case of an "infinite" stream?

[padenot]

If the ring buffer truly does just overwrite the indexes of a single pre-allocated memory block that have already been written what is the effective difference between HTMLAudioElement and AudioWorklet for the use case of an "infinite" stream?

Roughly, the Web Audio API, that provides AudioWorklet is optimized for low-latency audio and can be power-hungry, HTMLAudioElement is optimized for power efficiency, with higher audio output latency.

[guest271314]

Relevant to this PR, in light of

Does ringbuf.js support infinite input stream? #6

It's a ring buffer, so yes. The existing example is essentially that, writing a sine wave continuously to the buffer from the main thread and reading from the worklet thread.

is your perspective that use cases

• Implement infinite audio streams, Web radio stations

is a non-goal of ringbuf.js and this PR should be closed?

[guest271314]

@padenot

I have tested an infinite Opus stream using both opusenc and MediaRecorder with ServiceWorker.

The next part is creating a local radio station that serves the live stream over HTTP to be set at src of HTMLMediaElement.

Proof-of-concept https://bugs.chromium.org/p/chromium/issues/detail?id=1161429#c97.

@OP

95, #96 demonstrate that HTMLMediaElement plays Opus audio in OGG or WebM container for at least 24 hours.

Local radio station proof-of-concept

An HTML document on the local machine, Chromium, Web Audio API, opusenc, with a given language for the server.

For the output stream we use MediaStreamAudioDestinationNode passed to MediaStreamAudioSourceNode which we connect to AudioContext.destination.

We then can capture the sink-input with parec and pipe the stream through opusenc to produce the HTTP response that is the live stream (minus latency)

index.html

<!DOCTYPE html>
<html>
  <head>
    <title>Test infinite Opus stream</title>
    <style>
      body *:not(script) {
        display: block;
      }
    </style>
  </head>
  <body>
    <p>Test infinite Opus stream</p>
    <a
      href="https://bugs.chromium.org/p/chromium/issues/detail?id=1161429"
      target="_blank"
      >Issue 1161429: FLAC and Opus Audio Streams Stop Playing</a
    >
    <h1>Click</h1>
    <!-- one method for connecting file playback to the live stream -->
    <input type="file" accept="audio/*" />
    <script src="./script.js"></script>
  </body>
</html>

script.js

var ac, msd, osc, recorder, controller, source, gainNode, mediaStream;

const input = document.querySelector('input[type=file]');
input.onchange = async ({
  target: {
    files: [file],
  },
}) => {
  try {
    console.log(
      await new Promise(async (resolve) => {
        const absn = new AudioBufferSourceNode(ac);
        // const response = await fetch(url, {cache:'no-store'});
        const ab = await file.arrayBuffer();
        const buffer = await ac.decodeAudioData(ab);
        absn.buffer = buffer;
        absn.onended = () => {
          absn.disconnect();
          resolve('done playing track');
        };
        absn.connect(msd);
        absn.start();
      })
    );
  } catch (e) {
    console.error(e);
  }
};
// start the infinite local stream
document.querySelector('h1').onclick = async (e) => {
  const disconnect = async () => {
    console.log(e);
    msd.disconnect();
    msd.stream.getAudioTracks()[0].stop();
    osc.disconnect();
    await ac.close();
    return;
  };
  try {
    e.target.onclick = null;
    // microphone input
    mediaStream = await navigator.mediaDevices.getUserMedia({
      audio: {
        // avoid feedback from microphone
        echoCancellation: true,
        noiseSuppression: true,
        autoGainControl: true,
        latency: 0,
      },
    });

    ac = new AudioContext({
      latencyHint: 0,
    });
    // infinite stream of silence
    osc = new OscillatorNode(ac, { frequency: 0 });
    osc.start();
    msd = new MediaStreamAudioDestinationNode(ac, {
      channelCount: 2,
    });
    source = new MediaStreamAudioSourceNode(ac, {
      mediaStream,
    });
    source.connect(msd);
    gainNode = new GainNode(ac, {
      gain: 0.1,
    });
    osc.connect(gainNode);
    gainNode.connect(msd);
    const { stream } = msd;
    const [track] = stream.getAudioTracks();
    try {
      await track.applyConstraints({
        latency: 0,
      });
      console.log(
        await Promise.allSettled(
          ['echoCancellation', 'noiseSuppression', 'autoGainControl'].map(
            (constraint) =>
              track.applyConstraints({
                [`${constraint}`]: true,
              })
          )
        )
      );
    } catch (e) {
      // handle OverConstrained error
      console.log(e);
    }
    console.log(
      await mediaStream.getAudioTracks()[0].getConstraints(),
      await track.getConstraints()
    );
    // output we will capture with parec
    // now we can connect to and disconnect from MediaStreamAudioDestinationNode
    const o = new MediaStreamAudioSourceNode(ac, {mediaStream: msd.stream});
    o.connect(ac.destination);

  } catch (e) {
    console.error(e);
    disconnect();
  }
};

Get the sink-input (Chromium output) index

pactl list sink-inputs

Serve the live stream via HTTP

  if (isset($_GET["stream"])) {
    header('Vary: Origin');
    header("Access-Control-Allow-Origin: *");
    header("Access-Control-Allow-Methods: GET, OPTIONS, HEAD");
    header("Access-Control-Allow-Headers: Content-Type, Access-Control-Allow-Headers");    
    header("Content-Type: audio/x-opus+ogg");
    header("X-Powered-By:");
    // where 322 is the index noted from pactl list sink-inputs
    echo passthru("parec -v --raw --rate=48000 --monitor-stream=322 | opusenc --expect-loss=25 --max-delay=0 --framesize=2.5 --bitrate=256 --raw - -
");
    exit();
  }

ringbuf.js is not necessary to achieve the requirement.