search

brave / brave-browser

Brave browser for Android, iOS, Linux, macOS, Windows.
https://brave.com
Mozilla Public License 2.0
17.86k stars 2.33k forks source link

[Desktop] Support capture of monitor devices at Linux #11691

Open guest271314 opened 4 years ago

guest271314 commented 4 years ago

Description

Chromium refuses to support capture of monitor devices at navigator.mediaDevices.getUserMedia({audio: true}) https://bugs.chromium.org/p/chromium/issues/detail?id=931749.

Support capture of monitor devices at Brave browser.

Steps to Reproduce

  1. Execute navigator.mediaDevices.getUserMedia({audio: true})
  2. Execute navigator.mediaDevices.enumerateDevices() after navigator.mediaDevices.getUserMedia({audio: true})

Actual result:

Monitor devices are not listed at Linux.

Expected result:

Monitor devices to be listed at enumerateDevices() and capable of being captured by getUserMedia().

Reproduces how often:

Easily reproduced.

Brave version (brave://version info)

1.16.12 Chromium: 86.0.4240.30 (Official Build) nightly (64-bit)

Version/Channel Information:

Other Additional Information:

Miscellaneous Information:

guest271314 commented 3 years ago

@rebron What does "Chromium/waiting upstream" tag mean? The change requested in Brave in the exact opposite path of what Chromium authors have decided to take https://bugs.chromium.org/p/chromium/issues/detail?id=1155954. If that is an issue this bug should probably be closed.

bsclifton commented 3 years ago

@guest271314 sorry, I removed the waiting upstream tag as Chromium authors closed as wontfix

guest271314 commented 3 years ago

@bsclifton FWIW, a brief synopsis of relevant workarounds to capture entire system audio output, and specific playback devices https://github.com/guest271314/SpeechSynthesisRecorder/issues/17#issuecomment-744068599, https://github.com/guest271314/SpeechSynthesisRecorder/issues/17#issuecomment-749875748, https://github.com/guest271314/setUserMediaAudioSource. Notice that the Web has not broken becuase the user defined which devices on their machine they decide to capture with navigator.mediaDevices.getUserMedia().

guest271314 commented 3 years ago

@bsclifton A short list of Chromium FUGU requests that are still open Issue 1136480: FUGU Implement capture of system and application audio output to headphones and speakers https://bugs.chromium.org/p/chromium/issues/detail?id=1136480; Issue 1100101: Ability to create virtual microphone and camera devices https://bugs.chromium.org/p/chromium/issues/detail?id=1100101; Issue 1151308 in chromium: Implement Media Capture Automation https://bugs.chromium.org/p/chromium/issues/detail?id=1151308.

guest271314 commented 3 years ago

@bsclifton Importantly, note that this issue Default device selection and listing inconsistencies

727 https://github.com/w3c/mediacapture-main/issues/727 was closed based on this proffered criteria

The concept of default device is highly platform dependent and is not always defined, so the current language of the spec is probably the best we can do. There is no way to define a mechanism in the spec to determine default devices that works for all device kinds and all platforms. For example, Windows and Mac have very well defined default audio input and output devices, but they do not define any default camera device. Implementations have no problem using the default audio devices. Since there is no concept of default video input device, implementations may use different criteria to decide which device to list first in enumerateDevices. Since it is impossible to define a mechanism to determine default devices that works for all platforms and device kinds, I don't think there is much that the spec can do beyond the current language encouraging to use system defaults.

Edit: Remove closing this bug since I can't close bugs in this repo (got confused with mediacapture-output).

thereafter followed by the opening of this issue default audio output should be first in the enumerateDevice returned list #756 https://github.com/w3c/mediacapture-main/issues/756

The spec already mandates that the default microphone and the default camera (if they exist) should be the first devices listed in enumerateDevices. This should also apply to speakers.

This bug is basically the same as w3c/mediacapture-output#113, but I filed it here because enumerateDevices is defined in this spec.

If the former is true and correct the latter is moot due to 1)

There is no way to define a mechanism in the spec to determine default devices that works for all device kinds and all platforms.

and 2) "audiooutput" as described in W3C Media Capture and Streams is not a plain meaning "audiooutput" device that can be captured, rather is a term of art 'audiooutput' does not mean system audio output #18 https://github.com/guest271314/SpeechSynthesisRecorder/issues/18, thus is a misleading description of what occurs at Chrome - Mozilla browsers does not use "audiooutput" label Issue 1114422 in chromium: enumerateDevices() listing device kind "audioouput" is incorrect and misleading https://bugs.chromium.org/p/chromium/issues/detail?id=1114422#c9

"audiooutput" refers to audio playback via a media element. It does not refer to microphone input or audio capture of anything.

and capable of confusing users in the field that actually are intending to capture "audiooutput" with getUserMedia()

This doesn't record speaker output. I tried capturing tab audio using chrome extension but still failed. It seems speechSynthesis is not using HTMLmediaElement for audio hence we shall not be able to capture at tab/browser level. The audiooutput mentioned above returns "default " for both mic and speaker since there is no way to set "kind" field while setting constraints in getUsermedia, it always captures "mic". Let me know in case more details required. – Gaurav Srivastava Mar 4 '19 at 1:13 Confirming that it records from microphone rather than speech synthesis - at least in Chrome 84. – joe Aug 13 '20 at 11:15

Since Chromium is ostensibly FOSS, and Brave, AFAIK, is not bound to repeat verbatim what Chromium source code does, else there would be no reason for Brave, Brave browser is well-suited to implement at least the capture of monitor devices at getUserMedia() - though that actually does not go far enough to meet the use cases https://github.com/WebAudio/web-audio-api-v2/issues/106#issuecomment-751553951, particularly re capturing of Web Speech API, where multiple speech modules can each use a different playback device description (e.g., Google voices that are remotely requested use "Playback" device; on Chromium where no Google voices are shipped, for example, speech-dispatcher-espeakng can be the playback device; when both are used different devices will need to be captured to get each speech synthesis engine output to the exclusion of capturing all other playback devices that might be playing at the same time).