Are pointer actions low-level enough?

[RByers]

The pointer actions seem to be written in terms of dispatching DOM events. Does this level of abstraction really have much value for interop testing? I.e. it seems like you could get similar behavior by just creating the events directly in JS and dispatching them.

I think most of the interop testing value would come from defining synthetic input at a lower level of abstraction. I.e. instead of "dispatch pointerdown to element foo with pointertype=touch" I'd want "Add touch contact at window position 200,300". Then we can test behavior like touch scrolling, interaction of the various DOM event APIs (pointer, touch, mouse), behavior of CSS properties like touch-action, etc.

@summerlw @navidz @tdresser and @samuong have spent some time looking at a potential WebDriver API for input.

The actual pointer API defined here may still be (mostly) OK, the wording would just need to be changed to indicate that the action isn't to generate a pointer event. The action is to generate synthetic input which (among other things) has the effect of generating such a pointer event.

[samuong]

+1 from me too. This was discussed in the WebDriver F2F in Redmond last month (see https://www.w3.org/2016/07/13-webdriver-minutes.html#item03) and it seems that all the major vendors are doing some kind of synthetic input, but the difficulty is in how we word the requirement in the spec without being too "hand-wavy".

[RByers]

I see, Thanks Sam! Here's an example of a couple concrete questions (not hand-wavy) that I think the spec definition should somehow imply an affirmative answer:

• Is it possible to change the scroll position (fire scroll events etc.) via pointerdown/pointermove actions with pointerType=touch
• Is it legitimate for "pointer" actions to result in the generation of other DOM events (eg. mouse*, touch*, click).
• Is this API expected to have behavior on browsers which don't support DOM pointer events (eg. Safari)?

But anyway ultimately I don't care that much about the wording in the spec, as long as there's a consensus on the intention here (sounds like there was already) and tests can rely on this API for this sort of thing.

[samuong]

also cc @jgraham and @andreastt

[jgraham]

Yes, I think there is consensus that we want something that hooks into the browser at a layer above (or below, depending on how you imagine this) DOM events, and causes DOM events to be generated, as well as other changes in the browser state that would be produced by the actions, for example scrolling. But this layer is not a well defined thing, so it's not exactly clear how to specify it, and there are some things we don't want e.g. IMEs to pop up.

For mouse events, the pointer events spec actually covers this.

I guess my answer to "what about browsers that don't support part of the platform" is "those browsers should support that part of the platform".

[andreastt]

First of all, a better forum for this discussion would be the WG’s mailing list.

The biggest issue in wording the actions API is that we need to describe what “add touch contact at position 200,300” means by referring to existing web platform specs that describe the expected effects on the DOM. I think we all agree on how we would like to see this work, as most vendors would prefer to implement a tap action by inserting some kind of synthetic event into the browser’s event loop, instead of dispatching fake untrusted events.

But it’s also worth noting that the WebDriver specification explicitly says in the introduction that

Conformance requirements phrased as algorithms or specific steps may be implemented in any manner, so long as the end result is equivalent.

I interpret this as the spec describing the expected outcome more than it describes step-by-step how individual browsers should implement it. Or to rephrase, that a browser is free to take any steps it wants, for as long as it meets the contract of dispatching such and such events in such and such order.

[staktrace]

Or to rephrase, that a browser is free to take any steps it wants, for as long as it meets the contract of dispatching such and such events in such and such order.

Isn't this backwards though? Say you write a WebDriver test that uses this actions API. The test says "do a touch-tap at location (x,y)" and then checks to make sure it gets the proper sequence of events at that location - "touchdown, touchup, mousemove, mousedown, mouseup, click" or whatever. However, if I'm understanding you correctly, the contract of "do a touch-tap" is by definition only satisfied if it sends those exact sequence of events. So the test is redundant - as long as the browser is a conformant implementation of the actions API, the test must pass. Does that make sense, or am I misunderstanding something?