Clarify transform behaviour/interaction with element-capture

[hrgdavor]

I am wondering what is expected behaviour when we restrictTo an element but element is transformed, for example

• 100x100 px
• transform: scale(0.5)

• on screen it will be 50x50 for the user

  Is it specified/expected that 100x100 px resolution is captured in that case ?

From the demo of the red/green/blue boxes, transparency is not included in the recording, so my hope/expectation is that I can transform the element to make some space for some other ui elements, and still record in the restrictTo element's original resolution.

[markafoltz]

To clarify: what is "scale: 0.5" above? Is it a CSS rule applied to your element or are you scaling the element's rendering with devicePixelRatio?

[hrgdavor]

CSS: transform: scale(0.5)

[markafoltz]

Element capture is only guaranteed to capture the pixels on the screen after CSS is applied, so if the devicePixelRatio is 1.0, you would get a 50x50 pixel frame.

Chrome implements upscaling during capture when possible, so if you want , you could try passing a larger maximum width/height to getDisplayMedia() constraints.

[hrgdavor]

Ok, I was playing with the demos a bit, and I am getting the feeling of the "pipeline".

Looking through the lens of recording the screen, cropTo is about defining the capture area by looking at the element location/size.

If that was the only option, I would completely understand avoiding the direction of what view transitions are doing. And restricTo in my opinion it is beginning to overlap with viewTransitions. It is not only taking pixels from the screen (that is clearly shown with transparency gone with restrictTo). In that instance you are intercepting element's "image".

There is a logic in the recording code that was stopping the recording when I apply most of transforms... bot for some reason css: transform: scale(0.5) was working (recording was not interrupted).

I have seen this: https://github.com/screen-share/element-capture/pull/13#issuecomment-1597672326

At the moment, I see view transitions as an implementation artifact of Chrome, and not a requirement we should consciously add to the spec. I think that if we phrase the spec correctly, theoretical implementation may choose to render twice, or to force all other stacking contexts to become transparent, or to use any other approach - so long as the result is identical.

Maybe sbdy could help me get in touch with "view transitions" folks :) ... what they are doing seems to be more aligned with what I am desiring beyond this spec (element-capture). Whatever magic they are doing looks closer to my desire to capture element image (not ATM much interested in video). If they would allow getting the image of an element out on a canvas, or directly as png/jpeg by calling whetever they call to prepare for view transition.

[markafoltz]

View transitions help Web applications implement smooth cross-document animations, but doesn't really allow applications to capture those animations as a MediaStream.

Chrome's element capture implementation happens to use the same underlying code as View Transitions and we think sharing the same layout restrictions makes sense for other implementations as well (e.g., stacking context, no 3D transforms).

If you want to play around with View Transitions they have shipped in Chrome. You can find out more here:

https://developer.chrome.com/docs/web-platform/view-transitions

If you have feedback on View Transitions you can file an issue using the Feedback link in the spec:

https://drafts.csswg.org/css-view-transitions/

Good luck!

[markafoltz]

I don't think any clarifications to the specification are needed here, so @eladalon1983 can you close this out?

[eladalon1983]

I don't think any clarifications to the specification are needed here, so @eladalon1983 can you close this out?

Done.

@hrgdavor, please feel free to comment if you want to reopen for whatever reason.