Open garykac opened 8 years ago
RByers
commented
3 months ago Is it perhaps time to revisit this? CSSOM View spec has long re-defined MouseEvent coordinates as double, and that's what we've shipped in Chromium for a decade now (except that we still round for mouse events for web compat). The difference continues to cause confusion, like this recent twitter thread. Fractional coordinates are really critical for some polished UIs like touch-based dragging and drawing.
RByers
commented
1 month ago To be clear, the specific proposal here is to update the spec to take in the extension which has been in CSS OM View for the past 10 years. That is, make MouseEvent and MouseEventInit coordinates double not long.
Separately it's open for discussion what the spec should say (if anything) about when platforms may want to constrain themselves to sending only integer values. For the past 10 years, Chromium has had the MouseEvent accessors truncate the values to integers, while the PointerEvent (which derives from MouseEvent) accessors do not. This was done as a compat mitigation and may or may not still be necessary today. But personally I'd recommend we just encode the Chromium behavior in the UI Events spec since we know it to be web compatible (and without any real downside IMHO).
mustaqahmed
commented
1 month ago Hiding fractions in MouseEvent still seems necessary from compat perspective: Chromium's attempt to expose fractional coordinates only in click events failed 3 years ago when we switched click-like events to PointerEvent. I can immediately recall seeing https://www.openstreetmap.org/ failing with coordinate comparisons!
RByers
commented
1 month ago Thanks, makes sense! pointermove is the one that really matters IMHO. Not sure anything else does.
Copied from W3C Bugzilla: https://www.w3.org/Bugs/Public/show_bug.cgi?id=28029
Jukka Jylänki 2015-02-14 18:52:55 UTC
In e.g. the MouseEvent structure, the mouse xy coordinates are stored with the data type 'long', see here: http://www.w3.org/TR/DOM-Level-3-Events/#interface-MouseEvent .
There does not seem to be a clear documentation on how these coordinates relate to the value of window.devicePixelRatio (or I'm missing it). Looking at current implementation in Firefox, Safari and Chrome, the coordinate system that the fields clientX, clientY, screenX and screenY are specified in are the hardware-independent "CSS pixel" units meaning that they do not directly correspond to physical pixels on the display.
That is, testing on a Macbook Pro with 2880x1800 retina display, in each Firefox, Chrome and Safari, window.devicePixelRatio gives 2, and running this page
http://www.w3schools.com/jsref/tryit.asp?filename=tryjsref_event_mouse_screenxy
the extents of screenX and screenY range between (0, 0) and (1440, 900). In order to get the exact pixel coordinate where the mouse lies in, one must multiply the screenX and screenY parameters by window.devicePixelRatio, i.e. by a factor of 2 in this case.
However, since screenX and screenY (and all the other fields that represent mouse coordinates are of type 'long', this means that after multiplying, in both X and Y directions every second pixel is missed and only 25% of all pixels on the display are addressable by screenX and screenY.
4K resolutions are coming in fast, and I expect that browsers might use a window.devicePixelRatio of 4 on those displays. This would mean that only 1/16th or 6.25% of all hardware pixels would be addressable by the mouse event structure!
It looks like all the mouse coordinates throughout all events in the DOM specs should be passed as type 'double' instead of 'long' in places where the coordinates are represented in CSS pixels so that each hardware pixel will be addressable.
I wonder if I'm missing something here, and if this issue can be (has been?) resolved in some other way?
Comment 1 Boris Zbarsky 2015-02-14 19:26:10 UTC
We should probably add accessors with new names that return the actual coordinate as a double. Changing the behavior of the existing names is unlikely to be web-compatible.
Comment 2 Boris Gjenero 2015-02-15 05:19:51 UTC
Note that movementX and movementY values described in the pointer lock API are also affected: https://dvcs.w3.org/hg/pointerlock/raw-file/default/index.html#extensions-to-the-mouseevent-interface
There is the same loss of precision as with position plus there is also the possibility that slow movement gets rounded to zero and ignored. It may be possible to slowly move the mouse as far as you want without causing any movement to be reported. I can reproduce this in Firefox 35 when using increased layout.css.devPixelsPerPx. A browser could work around this by accumulating sub-pixel motion, but floating point values are a better solution.
Comment 3 Anne 2015-02-15 08:43:30 UTC
Note that the new iMac with a 5K screen simply uses a ratio of 2 so it's not that big of deal yet, but we should probably do something now so it's deployed and ready a couple years from now.
Comment 4 Jukka Jylänki 2015-02-15 10:06:25 UTC
I would be very cautious to state that this is not currently a big deal, since already now only one quarter of the pixels on the screen are representable by the DOM events on high DPI displays, and Macbook Pro retina displays came out already in 2012.
I believe this issue also exists for all touch events http://www.w3.org/TR/touch-events/#touch-interface , and currently the majority of mobile devices out there have a window.devicePixelRatio greater than one. For example, Google Nexus 4 has a devicePixelRatio of 2, and iPhone 6 Plus has a devicePixelRatio of 3.
Also like Boris Gjenero mentions, this issue causes precision and stuttering to pointer lock and relative mouse movement.
In my opinion, this is definitely a critical P1 bug. Most web APIs do not properly handle high DPI, and the invention of hardware-independent pixels was not done with a good understanding or design of its wide-spanning implications on the different web APIs. This is a surface area of bugs that keeps popping up in different places, here's a small example of recent issues to illustrate:
https://github.com/KhronosGroup/WebGL/issues/587 https://www.khronos.org/webgl/public-mailing-list/archives/1406/msg00019.html https://bugzilla.mozilla.org/show_bug.cgi?id=1021547 https://bugzilla.mozilla.org/show_bug.cgi?id=1024493
And the common underlying issue in all of these is that the web APIs were not designed to account for window.devicePixelRatio > 1.
My point is that everything is already broken now, and not just in a few years. I would like to plea that this issue is taken seriously in W3, and even further, I would like to see that non-high DPI APIs would get aggressively documented as being broken and deprecated in the future.
Comment 5 Simon Pieters 2015-02-16 19:05:28 UTC
http://dev.w3.org/csswg/cssom-view/#extensions-to-the-mouseevent-interface specifies MouseEvent's members as double. This has been shipping in IE without opt-in since IE10. Blink plans to implement it also, see http://crbug.com/456625
Also see this thread: https://lists.w3.org/Archives/Public/www-style/2015Feb/thread.html#msg193
Comment 6 Travis Leithead [MSFT] 2015-02-16 20:05:36 UTC
(In reply to Simon Pieters from comment #5)
Note, we just reverted this behavior in our latest IE builds due to compat issues (with web content that works on Chrome). Specifically, for MouseEvent instances, IE will no longer return coordinate members as double. (We still do for Touch/Pointer though.)
Comment 7 Simon Pieters 2015-02-17 09:44:52 UTC
Thanks Travis. Do you have information on which URLs broke? It is sad that MouseEvent has not been prioritized for Blink before you had to revert it.
Comment 8 Travis Leithead [MSFT] 2015-03-12 22:57:42 UTC
(In reply to Simon Pieters from comment #7)
Looking back through the checkins... looks like:
Comment 9 Simon Pieters 2015-03-16 11:18:30 UTC
OK, that's the same as https://www.w3.org/Bugs/Public/show_bug.cgi?id=24159#c2
Do I understand correctly that there were no other sites reported broken? Would you consider implementing it again if Blink implemented it?
Comment 10 Arthur Barstow 2015-03-21 15:11:54 UTC
Bulk move of all D3E bugs to the UI Events component.