Chromium now supports capture of monitor devices using getDisplayMedia()

[guest271314]

The sad part of the story is that Chromium developers deliberately chose to refuse to list or capture Monitor devices on Chromium (see here and here).

See https://issues.chromium.org/issues/40155218.

Some caveats

• Behind flag #pulseaudio-loopback-for-screen-share.

• WebRtcAllowInputVolumeAdjustment feature is enabled by default. Decreases monitor volume from 100% to 8%.

• Does not capture system audio when Tab is selected in Screen Capture picker.

(Does not have a means capture specific sink inputs, e.g., speech-dispatcher-espeak-ng, and filter other devices; just happens to capture Web Speech API output, not the exclusion of other devices or playback, e.g., Chromium playback in a Tab.)

Tested capturing speechSynthesis.speak() on Chromium Version 123.0.6284.0 (Developer Build) (64-bit) https://gist.github.com/guest271314/baaa0b8d4b034ff4e9352af4f2bbf42c.

[edisionnano]

That's interesting, I should correct the md. The problem with Discord is that you don't want Discord audio in the stream (which the monitor inevitably captures) but it can be useful for other purposes outside Discord Screenshare.

[guest271314]

I should correct the md.

That's my sole reason for filing this issue. Accurate documentation.

[edisionnano]

I should correct the md.

That's my sole reason for filing this issue. Accurate documentation.

Yeah thank you, I corrected it. The biggest issue remaining now is stereo audio, even if Chromium is forced to capture stereo through getDisplayMedia flags Discord will still downmix it to mono. I have a suspicion that this happens on the client but haven't delved into it.

[guest271314]

I have not done any audio processing in Discord. We can definitely create and record two (2) channels client side with these constraints using getDisplayMedia() or getUserMedia() when we create a virtual remapped device with AudioWorklet https://github.com/guest271314/MP3Recorder or MediaStreamTrackProcessor https://github.com/guest271314/MP3Recorder/blob/media-capture-transform/MP3Recorder.js

let stream = await navigator.mediaDevices.getDisplayMedia({
  // We're not going to be using the video track
  video: {
    width: 0,
    height: 0,
    frameRate: 0,
    displaySurface: "monitor"
  },
  audio: {
    suppressLocalAudioPlayback: false,
    // Speech synthesis audio output is generally 1 channel
    channelCount: 2,
    noiseSuppression: false,
    autoGainControl: false,
    echoCancellation: false
  },
  systemAudio: "include",
  // Doesn't work for Tab capture
  // preferCurrentTab: true
});

[edisionnano]

I have not done any audio processing in Discord. We can definitely create and record two (2) channels client side with these constraints using getDisplayMedia() or getUserMedia() when we create a virtual remapped device with AudioWorklet https://github.com/guest271314/MP3Recorder or MediaStreamTrackProcessor https://github.com/guest271314/MP3Recorder/blob/media-capture-transform/MP3Recorder.js

let stream = await navigator.mediaDevices.getDisplayMedia({
  // We're not going to be using the video track
  video: {
    width: 0,
    height: 0,
    frameRate: 0,
    displaySurface: "monitor"
  },
  audio: {
    suppressLocalAudioPlayback: false,
    // Speech synthesis audio output is generally 1 channel
    channelCount: 2,
    noiseSuppression: false,
    autoGainControl: false,
    echoCancellation: false
  },
  systemAudio: "include",
  // Doesn't work for Tab capture
  // preferCurrentTab: true
});

Yeah channelCount: 2 is what I foced, I haven't tried systemAudio or suppressLocalAudioPlayback but I remember verifying that the audio being sent from chrome was indeed stereo by capturing it to a file using JS. Discord receives said stereo stream and makes it mono, or picks one channel I don't remember. What I don't know is if said downmix happens on the client side or server side.

[guest271314]

Discord receives said stereo stream and makes it mono

What I don't know is if said downmix happens on the client side or server side.

Has to happen in Discord world per the evidence above.

I've only very moderately used Discord Web interface.

[edisionnano]

Discord receives said stereo stream and makes it mono

What I don't know is if said downmix happens on the client side or server side.

Has to happen in Discord world per the evidence above.

I've only very moderately used Discord Web interface.

Yeah it's a Discord thing, I need a way to capture the WebRTC packets to see if it happens on the client but I don't know how

[guest271314]

You should be able to log all WebRTC statistics in Chrome (and Firefox).

[edisionnano]

You should be able to log all WebRTC statistics in Chrome (and Firefox).

I can check chrome://webrtc-internals/ but there is still a chance the downmix or channel picking happens after that in JS before it gets to the server

[guest271314]

Why not just exchange SDP in Discord and create your own communication channels.