screen-share / element-capture

https://screen-share.github.io/element-capture/
20 stars 6 forks source link

Proposal: "Replace" pixels instead of "remove" #10

Closed JonnyBurger closed 1 year ago

JonnyBurger commented 1 year ago

Referring to this phrase:

Remove any pixels not derived from either the target-element or its descendants. (TODO: This point requires a less hand-wavy definition.)

So, let's say that a outside menu or dropdown is obscuring the target-element. Instead of removing those pixels (and leaving black spots), it would be nice if the browser could record the pixels underneath.

If so, we could maybe say something like:

Pixels that come from elements that obstruct the target element or its descendants should be replaced by pixels of that would have been drawn if the obstructing element was invisible.

I can't judge if this is practical or makes the implementation much harder, but I wanted to bring it up.

eladalon1983 commented 1 year ago

Capturing the pixels underneath is indeed what this API intends to do. Leaving black spots was never the intention, nor do I think it follows from the current phrasing (black pixels are not a valid sentinel value). See this text from the explainer:

The Element Capture document introduces a new mutation mechanism which we name "restriction". After a Web application "restricts" a video track to a given target-element, frames produced on that video track only consist information from the target-element and (and its descendants). Phrased differently, the track becomes a capture of the DOM sub-tree rooted at the target-element.

The spec itself doesn't explain things accurately enough - hence the TODO it has. :-)