obsproject / obs-browser

CEF-based OBS Studio browser plugin
GNU General Public License v2.0
771 stars 218 forks source link

Add audio input for getUserMedia #357

Closed oshiteku closed 2 years ago

oshiteku commented 2 years ago

Description

Adds an audio input property to allow the browser to retrieve audio from other sources. Hook getUserMedia when the browser starts loading to convert the audio data from obs into a MediaStream.

screenshot: Screen Shot 2022-03-21 at 4 21 15

Motivation and Context

This feature allows web applications to create widgets that analyze and visualize OBS audio.

Since JavaScript has Web Audio API and WebGL, a very wide variety of applications could be realized with this feature.

How Has This Been Tested?

I have confirmed that it works with macOS 12.3 and Apple Silicon. Tested with several web applications that use getUserMedia.

Tested web applications:

Types of changes

Checklist:

WizardCM commented 2 years ago

This will break existing uses of getUserMedia performed by launching OBS with --enable-media-stream --use-fake-ui-for-media-stream.

Additionally, the CEF build we include as part of OBS 27.2 includes a PR for CEF which allows us to control access to getUserMedia natively. https://bitbucket.org/chromiumembedded/cef/pull-requests/444 I plan on pushing the UI changes to allow users to control this in the coming months. Currently (in my branch) it's a pretty basic dropdown that lets the user choose what system devices the page has access to, microphones, webcams, desktop audio, or screen capture.

WebpageAccessLevel="Hardware Permissions"
WebpageAccessLevel.Level.None="No access to hardware"
WebpageAccessLevel.Level.Audio="Allow access to audio (microphones, etc)"
WebpageAccessLevel.Level.Video="Allow access to video (webcams, displays, etc)"
WebpageAccessLevel.Level.AudioVideo="Allow access to both audio and video"

That said, I do think providing an option to send data specifically from an OBS source would be incredibly useful - I wonder if it could be done using the media access handler API (from that CEF PR) rather than overriding the built-in getUserMedia function.

oshiteku commented 2 years ago

Thanks for the information. Then this PR approach is not appropriate.

I plan on pushing the UI changes to allow users to control this in the coming months. Currently (in my branch) it's a pretty basic dropdown that lets the user choose what system devices the page has access to, microphones, webcams, desktop audio, or screen capture.

This is what I wanted, so I'm closing this pull request.

That said, I do think providing an option to send data specifically from an OBS source would be incredibly useful - I wonder if it could be done using the media access handler API (from that CEF PR) rather than overriding the built-in getUserMedia function.

That sounds like a good idea. If I come up with a better way, I will make another pull request.