w3c / mediacapture-screen-share-extensions

Other
1 stars 0 forks source link

Avoid user-confusion by not offering undesired audio sources #7

Open eladalon1983 opened 2 years ago

eladalon1983 commented 2 years ago

User agents may offer audio to be captured alongside video, if the application specifies {audio: true}, or maps audio to anything else that's different from false. But not all audio is created alike. Consider video-conferencing applications. Tab-audio is often useful, and can be shared with remote participants. But system-audio includes participants' own audio, and it is NOT desirable to share it back.

State of the art? VC applications can ask for "audio", use it if it's tab-audio, and discard it otherwise. This works, but it's sub-optimal. The user is left confused. The user wanted to share system-audio, the user was offered to share-system, the user explicitly approved sharing system-audio - and now remote participants are telling the user that they can't, in fact, hear the system-audio. Now how messed up is that?!

It'd be better if we allowed the application to ask for less. Allow the application to ask for audio only in conjunction with surfaces of type X. (For example, with constraints.)

I think we should debate this in order:

  1. Desirability. Are we in agreement that it is desirable to allow the application to ask for less?
  2. Mechanism. Provided that we agree on no1, how do we want to shape the control surface?
eladalon1983 commented 2 years ago

My understanding of today's WebRTC Working Group meeting was that the suggestion in this slide was generally accepted, with some bikeshedding left over names. I propose that we reuse displaycapturesurfacetype but name the key desiredSurfaces and make it a set. I am fine with adding @jan-ivar's desired text over rejecting the set ["monitor"] or ignoring it. Wdys? @jan-ivar? @youennf?

youennf commented 2 years ago

Proposal from editor's meeting: add something like 'boolean avoidSystemAudio' member field in https://w3c.github.io/mediacapture-screen-share/#dom-displaymediastreamconstraints. default value is false (allowing system audio opt-in by user)

eladalon1983 commented 2 years ago

The brainstorming session continued and we landed on:

enum SystemAudioPreferenceEnum {
  "include",
  "exclude"
};

dictionary DisplayMediaStreamConstraints {
  SystemAudioPreferenceEnum systemAudio;
};
eladalon1983 commented 9 months ago

Expectedly, some Web applications (Meet) are now asking for the ability to exclude both window- and system-audio, and only prompt the user for tab-audio (alongside video, of course). I think it's always good for browsers to get out of the applications' way when they wish to ask for less permissions, so I propose we follow suit on systemAudio "include"/"exclude" and introduce windowAudio as "include"/"exclude" too.

Namely:

enum WindowAudioPreferenceEnum {
  "include",
  "exclude"
};

dictionary DisplayMediaStreamConstraints {
  WindowAudioPreferenceEnum windowAudio;
};

And text saying that {windowAudio: "exclude", systemAudio: "include"} is illegal and results in rejection, in order to prevent users being nudged towards sharing their entire screen.

Wdys?

-- CC @o1ka.