Virtual device registration for MediaStream and friends

[hoch]

There was an idea floating around to integrate MediaStream to AudioDeviceClient. This is just a straw-man code example.

// These are MediaStream objects that are already opened.
const remoteConsumer;
const remoteProducer; 

// Scenario: device #6 and #7 are remote input and output devices respectively.
(async () => {
  await navigator.mediaDevices.registerAudioSink(remoteConsumer);
  await navigator.mediaDevices.registerAudioSource(remoteProducer);

  const devices = await navigator.mediaDevices.enumerateDevices();  
  const constraints = {
    mode: ‘raw’,
    outputDeviceId: devices[6].deviceId,
    inputDeviceId: devices[7].deviceId,
    callbackBufferSize: 2048,
    inputChannelCount: 2,
    outputChannelCount: 2,
  };

  const client = await navigator.mediaDevices.getAudioDeviceClient(constraints);
  await client.addModule('my-client.js');
  client.start();
})();

This needs a spec work for MediaDevices API, which might take a while. However, once we can make it the API shape looks quite sensible.

cc @alvestrand

[chrisguttandin]

If I understand it correctly the Screen Capture spec does support to get a MediaStream with audio from other tabs or windows. Although it seems to be not supported by any browser so far. Would it make sense to allow other tabs or windows as an input or output of an ADC as well? This would allow to build something like Soundflower in the browser.

[padenot]

This is best done as an API to route a MediaStream to an ADC.

A MediaStream is already a good opaque object to represent a stream of real-time media, no need to reinvent the wheel.