MouseEvent.screenX/Y coordinate space is not clear

[NavidZ]

@bokand @smaug---- @garykac @RByers @dtapuska @eirage

It is not very clear what the coordinate space of screenX/Y attributes is. Is that the correct that is it in css pixels? In other words if screenX/Y of the events is increased by 1 does that mean it has moved 1 css px? So if I have a full screen browser and I set the left/top of a div to the screenX/Y of events should it follow the mouse pointer (as in this example)?

Note that I don't care about the offset as you can move the browser window in the screen. I care about the scale of each pixel.

[NavidZ]

Anyone knows the answer?

[bokand]

I'm not an expert here but according to https://drafts.csswg.org/cssom-view/#dom-mouseevent-screenx, it seems that this should be in CSS pixels (since "Web-exposed Screen Area" is defined in CSS Pixels) though that's still an ED. That seems to match both Firefox and Chrome's implementations. In general, all coordinates I know of in the browser are in CSS pixels. Though for screenXY it does seem a bit awkward as the rest of the screen doesn't have browser zoom applied to it...

[bokand]

Sorry, I have it backwards above - it appears that browser-zoom doesn't affect screenX/Y in practice so they're not in CSS pixels

[NavidZ]

So I assume the goal was to have screenX/Y match the css pixels. Particularly both Chrome and Firefox do that when dsf changes but none of them do that when page zoom (i.e. browser zoom) changes. However, changing either dsf or page zoom in both browsers will change window.devicepixelratio. @bokand, do you think that is a reasonable approach to have page zoom match the dsf behavior here? @smaug----, what do you think since Firefox does this too?

[smaug----]

(It is Firefox, not FireFox ;) )

https://bugzilla.mozilla.org/show_bug.cgi?id=809788 has some background information. See comment 26 for example.

[bokand]

Looking at the history on that bug and roc's thread on www-style, it seems that we eventually aligned with Firefox on how devicePixelRatio should behave. That is, ctrl+/- does affect devicePixelRatio. I have tried this out and Chrome, Firefox, and Safari all share the same behavior. (according to www-style, IE did as well so I assume it still does)

I think the prevailing interpretation on the thread (which I agree with) is that ctrl+/- zoom and the zoom applied for high-dpi displays is effectively equivalent. Given that, I do think it's a bug that screenX/Y take into account one but not the other. I don't have a strong opinion on whether it should be in physical pixels or in CSS pixels but it seems like a bug that we're treating DSF differently from ctrl+/- zoom.

[RByers]

I agree we want DSF and browser zoom treated the same. Assuming it's web compatible I'd prefer if screenX/Y were in CSS pixels for consistency with other APIs (it's rare for a developer to need to think in physical pixels). Of course what's most web compatible and interoperable is probably what will primarily determine what exactly we need to do here.

[NavidZ]

Just to get everyone on board with this. @smaug----, @patrickkettner do you also agree having the screenX/Y in CSS pixels is okay (assuming there isn't many compat issues)? Particularly what I have in mind is changing the page zoom logic to also change screenX/Y range similar to what device scale factor does.

[smaug----]

It is now in CSS pixels, but sounds like your asking about zoom behavior. How should things like window.inner* work, or window.screenX/Y work? Changing any of this is rather regression prone. What is the issue you're trying to solve?

(Is dsf commonly used acronym?)

[bokand]

If the property doesn't react to ctrl+/- zooming then it isn't in CSS pixels since that increases/decreases the size of a CSS pixel. Other properties like window.innerWidth do change as the user zooms in and out.

DSF = device scale factor. It's the zoom applied by the browser to compensate for a high pixel density display (i.e. so that the UI doesn't look tiny). That's what we call it on Chrome, not sure what other browsers call it.

We should be careful about compat impact but in this case, since we're already applying the DSF to these properties, content that will break wit this change would already have been broken on high density displays so I suspect regressions would be rare. Users could work-around by resetting zoom level so any issues would be low severity IMO.

[smaug----]

Are mouseevent.screenX/Y the only properties which don't scale?

In general I do support the change, but I would assume some pages will be broken. Like, they could currently try to convert mouseevent.screenX/Y to something reasonable using other coordinates like clientX/Y and properties of screen object.

[scheib]

Would someone research & post the current state of implementations?

[mustaqahmed]

@EiraGe has created an extensive report on PointerLock coordinate space where we observe that all major browsers except Edge expose screenX/Y coordinate in DIPs (device independent pixels). So changing the screenX/Y spec to match the reality seems to be the best way forward for this issue.

@smaug----, does it sound like an acceptable solution here?

This is remotely related to this issue but we discussed it here: we also find in the above doc that we currently have no factors (in spec or in browsers) that allows conversion between DIPs and physical/CSS pixels. We will handle it separately, possibly through the PointerLock coordinate thread. (Note that devicePixelRatio is between physical/CSS pixels.)

[garykac]

Relevant table from @EiraGe's document:

ScreenX/Y	Edge	Chrome	Firefox	Safari	ChromeMobile	Firefox Mobile	Safari Mobile
coordinate space	Physical pixel	DIP	DIP	DIP	DIP	DIP
Scaled by device-scale-factor(OS High DPI setting)	X	✓	✓	✓	✓	✓	✓?
Scaled by browser zoom	X	X	X	X	No browserzoom		No browserzoom
pinch-zoom-scale	X	X	No pinch zoom	X	X	✓(scale)	✓(origin & scale)

(where PhysicalPixel * DeviceScaleFactor = DIP and DIP * BrowserZoom = CssPixel)

With the recent announcement of Edge moving to adopt Chromium/Blink, I assume that Edge will soon be using DIPs as well. If so, then updating the spec to specifying screenX/Y as DIPs seems noncontroversial.