search

w3c / mediacapture-screen-share

Media Capture Screen Capture specification
https://w3c.github.io/mediacapture-screen-share/
Other
85 stars 28 forks source link

Provide a means to select only part of a screen to capture #105

Closed guest271314 closed 5 years ago

guest271314 commented 5 years ago

Firefox provide a "Take a Screenshot" feature which allows the user to select only a portion of a screen. That option should be provided for getDisplayMedia() in the form of constraints where specific coordinates can be passed; e.g., using .getBoundingClientRect() in the form of {topLeft:<pixelCoordinate>, topRight:<pixelCoordinate>, bottomLeft:<pixelCoordinate>, bottomRight:<pixelCoordinate>}, or at the selection UI, similar to how Firefox implements "Take a Screenshot" feature. Use case: The user only wants to share a specific element, e.g. a <video>.

guest271314 commented 3 years ago

@danjenkins

Ultimately this shouldn't be a userland addition. This should be a browser feature.

Do you really trust browser implementations to that degree? I do not. It took 2 years just to convey the reality that Chromium was consistently crashing when variable pixel dimension frames were decoded.

Re the arbitrary restriction on Picture-In-Picture window implemented by Chromium, see this response https://github.com/w3c/picture-in-picture/pull/186#issuecomment-646155752

@guest271314 please stop asking the same question over and over again because you do not like the answer.

This language is a spec RECOMMENDATION which is in no way a requirement but a suggestion. This suggestion comes from the experience of platforms and browsers implementing Picture-in-Picture (either the Web API or another sort). If a browser implementing this API wants to do something else, it's fully up to them.

Lets say I only want to share the top half of my vertical monitor. I have something private and confidential on the bottom half that I haven't moved off it - there's no way I'm giving say Google Meet access to my entire screen, would I trust that application to not be sending that data somewhere?

Process the frames for yourself to ensure you are only exposing what you expect. Again, the PR that fixes exposure of prompt (potentially including windows, applications not intended to be captured) at first frame is not fixed in Chromium source code.

What you're describing above feels like a hack around a solution - and I'm not even certain if its really fulfils the solution to the problem - what if I don't have the browser foregrounded at all.

Well, that is all we have right now. Feel free to create a repository specifically for what your goal is and will help where able.

I'm showing off VsCode and another browser different to the one running the meeting application I'm sharing my screen with....

Would explore using "Application" capture at Chromium.

In the web world we stop HTTPS pages being able to talk to HTTP APIs for a reason....

There are various way to get around that limitation, e.g., https://gist.github.com/guest271314/04a539c00926e15905b86d05138c113c.

how is this any different? You want the user to know the web application truly only has access to the portion of the screen I allowed it access to. Also, the above solutions only work if developers implement the ability to do it - shouldn't it be a browser GUM "thing" so that any web application taking advantage of getDisplayMedia is able to use it?

Agree that the specification could solve the problem, though the issue was closed. Thus, here, Fix WontFix on own, without waiting for anyone else.

guest271314 commented 3 years ago

@danjenkins As an example of what could be fixed by simply specifying capture of monitor devices MUST be the same as capturing only microphone devices, created an entire repository with multiple solutions, where that same subject matter is applicable to this specification https://github.com/w3c/mediacapture-screen-share/issues/140, where audio capture is described in an ambiguous manner https://github.com/guest271314/captureSystemAudio.

danjenkins commented 3 years ago

@guest271314 We need the browser to be able to do it so that all webrtc applications, no matter whether the developer wants to, can offer this.

How can we get this re-opened?

dontcallmedom commented 3 years ago

@danjenkins if by "it" you mean the browser should allow to capture a subset of a given screen, I believe nothing in the spec prevents it, and it's not clear that there is anything the spec can say to require it since ultimately the user-selection mechanism of what gets shared is left to browsers.

Has any filed bugs with browsers to get this new display selection option implemented? Until it is clear that the spec needs to change, it's not obvious there would be much point in reopening the issue.

danjenkins commented 3 years ago

@dontcallmedom thanks for this - yes I meant the browser - I still wasn't 100% on whether the spec allowed for this or not. Thanks for the clarification! I don't know of any bugs filed yet but I must admit I'm not the best at searching crbug and co. I'll go make a crbug and mozilla bug entry and go from there I guess :) thanks!

guest271314 commented 3 years ago

@danjenkins Do not recollect filing Chromium and Mozilla bugs for this. For Mozilla bug would suggest incorporating the already existing screenshot code https://github.com/w3c/mediacapture-screen-share/issues/105#issuecomment-521681433 into the feature request. That is, similar to the overlay Chromium uses for getDisplayMedia(), the user is able to initially select a rectangle on the screen that will be captured for the duration of the stream. Relying on x, y programmatically alone will need to account for the navigation and title bars, which could be up to 30 pixels, if recall accurately when testing, thus used window.open(), CSS, and requestFullScreen() to remove those browser GUI parts from the equation.

guest271314 commented 3 years ago

@danjenkins

I still wasn't 100% on whether the spec allowed for this or not.

https://github.com/w3c/mediacapture-screen-share/issues/105#issuecomment-522012790

See the current language in the specification, emphasis added

Abstract This document defines how a user's display, or parts thereof, can be used as the source of a media stream using getDisplayMedia, an extension to the Media Capture API [GETUSERMEDIA].

abacaj commented 3 years ago

@danjenkins

I still wasn't 100% on whether the spec allowed for this or not.

#105 (comment)

See the current language in the specification, emphasis added

Abstract This document defines how a user's display, or parts thereof, can be used as the source of a media stream using getDisplayMedia, an extension to the Media Capture API [GETUSERMEDIA].

Is there a way to record a region of interest today using get user media api and media recorder?

shaochuancs commented 2 years ago

If the specification states that getDisplayMedia() can acquire "or part thereof" user's display, but the API design in specification does not allow to do so, it doesn't make any sense. It's even weird that this bug report of specification is rejected.