Clarify what touch-action allows with rotated scrollable elements

[staktrace]

I think the spec is unclear on what the intended behaviour is for elements which have rotation transforms on them. For example on this test page Chrome allows scrolling while your finger is moving in the "x" direction, even though that means the element is scrolling vertically (the scrollTop value is changing). While that's a reasonable thing to do, our implementation in Firefox does the opposite (allows scrolling such that scrollLeft changes).

[patrickhlauke]

The spec currently says

In the case of pan-left, pan-right, pan-up and pan-down, the direction is interpreted as the opposite of the physical movement in the screen co-ordinate space.

wondering if dropping the "In the case..." bit, and simply stating that "The direction..." relates to the (document-level?) screen co-ordinate space, would help clarify the situation here? (this, would then also explain Chrome's behavior)

[RByers]

Thanks for reporting this! We discussed this a bit here but I was sloppy. There's two things being conveyed by this sentence:

1. All directions should be interpreted relative to the "screen" co-ordinate space.
2. Up/down/left/right represent the opposite direction of finger travel (i.e. typically the direction in which panning would occur ignoring rotated iframes).

IIRC we went through a few edits of this text trying to come up with something concise that was accurate. Clearly I missed the non-direction-specific case. More importantly IMHO, we failed to add a test case for this. Let's use this bug to track fixing both.

[staktrace]

I guess we can discuss it F2F but I just tried this on Edge as well and it does the same thing that FF does (i.e. different from Chrome).

[RByers]

We discussed this at the hackathon, and realized that the Edge/Gecko behavior really is necessary. Imagine a page with a carousel that must allow horizontal dragging (so pan-y) hosted in a rotated iframe. The carousel will no longer work. So screen co-ordinate space really is wrong.

What is the right co-ordinate space then, is it client? I.e. should a CSS rotated DIV also behave like a rotated iframe?

[RByers]

Note that fully supporting this behavior (eg. arbitrary angle rotation) will be hard in Chrome's architecture (I don't think we handle non-axis-aligned scrolling well at all). In particular, we filter our gestures for touch-action in the browser process before we know anything about the content under the finger. But I think we could probably do a quick hack for the 90/180/270 rotation cases (take rotation into account when computing the effective touch action) and leave the rest as a pretty low priority bug blocked on our compositor hit-test rework.

[patrickhlauke]

What is the right co-ordinate space then, is it client?

document perhaps? (assuming that the iframe's content counts as its own document)

I.e. should a CSS rotated DIV also behave like a rotated iframe?

I'd say no, but I can see how that may be confusing. But if we said that it's document, it should be clear (maybe with extra clarification) that this is unaffected by any rotation etc.

[staktrace]

I would rather be consistent between divs and iframes, so use "client" coordinate space. I'm not sure if there's any value to allowing off-kilter scrolling (i.e. what Chrome does now if you have a div rotated to 45 degrees). Also, the argument for iframes (that a carousel hosted in a rotated iframe should behave sanely) applies to divs as well, in that somebody might be using a library to build a carousel and then want to host it inside a rotated div.

[patrickhlauke]

Would be good to get something roughed out for this (likely as part of the overall touch-action definition, rather than repeating it for each directional value)

[mustaqahmed]

We also need to fix the computation of effective touch-action:

A touch behavior is supported if allowed by the touch-action properties of all elements between the hit tested element and it's nearest ancestor with the default touch behavior... Perhaps defining the computation "recursively" is the simplest option, after mentioning the coordinate space above it.

[mustaqahmed]

Here is the Chrome bug.

[mustaqahmed]

Let's leave this open:

• The part on touch-action value changes during an ongoing action may need some attention.
• We need to add tests, better to track it here.

[mustaqahmed]

While rearranging the touch-action section (Sec 9) of the spec to bring all coordinate space related text close together (in my last pull request above), I also moved the Note on dynamic changes to touch-action values into a non-Note para (the third bullet in Sec 9.2). I will request a closer look just in case.

[patrickhlauke]

moving the dynamic changes bit out into an actual bullet looks good to me.

[RByers]

I would rather be consistent between divs and iframes, so use "client" coordinate space.

I think this is normally called "local" co-ordinate space, right? I.e. this is NOT what clientX and clientY events do (they are unaffected by rotated DIVs, just rotated iframes) IIRC. I think the non-standard MouseEvent.layerX/layerY properties may behave this way.

Can someone verify what Edge and FF do in the case of a rotated DIV? Does touch-action operate in client or the element-local co-ordinate space?

In particular if Edge and FF are both really using the client co-ordinate space today then I'd rather spec/test it that way. Yes in theory there could be a problem within a frame, but that problem exists for co-ordinates generally - the code rotating something needs to also apply the same rotation to any client co-ordinate handling (and can probably do so because it's all in the same frame). Only the iframe case is really unsolvable IMHO (because the code that knows about the transform in the parent frame doesn't necessarily have access to the events being delivered to the child frame). Implementing the local version is probably harder for chromium than implementing the client version. But if at least one other browser has really already implemented local as @staktrace described, then I'm OK saying Chrome just has a bug here for now.

[RByers]

Oh also we should think about how we can best test this. The touch-action tests today aren't very scalable (one test per gesture - a lot of boilerplate to add a few new test cases like the ones discussed here). I wonder if now is the time to rewrite them with an assumption of having automation driving them? Or at least using some generic framework in a single (or few) test file which can either be automated or driven manually one gesture after another in the same test... Thoughts?

[mustaqahmed]

Can someone verify what Edge and FF do in the case of a rotated DIV? Does touch-action operate in client or the element-local co-ordinate space?

For all of Edge, FF and Chrome, a rotated div filters touch-actions the same way as a rotated frame. Here is a repro that has "pan-y" in a rotated div.

Edge: Works perfectly: only horizontal (along screen x-axis) swipes can scroll the div along local y-axis direction. FF: Semi perfect. Horizontal (along screen x) swipes scroll the div along local y-axis. But some of the vertical (along screen y) swipes also seem to scroll the div along local y-axis, perpendicular to the swipe! Attention @staktrace. Chrome: Bad, ignores the rotation.

[mustaqahmed]

Looks like @staktrace's repro link in the original post is also for a rotated div. Then so far we don't know what happens to a rotated iframe, right?

[mustaqahmed]

Here is a rotated subframe repro. Looks like I was (blindly) right above: div and subframe behave similarly for each of the three browsers.

[RByers]

Thanks Mustaq! Ok then, documenting what Edge is already doing (and Firefox is mostly doing) seems indeed to be the way to go. Thanks for double checking!

[mustaqahmed]

The test has got merged.