Changing the DOM hierarchy while handling a "pointerenter" event produces significantly different results across browsers

graouts commented 5 years ago

Consider this example (which GitHub wouldn't let me attach):

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="utf-8">
    <title>Changing the DOM hierarchy while handling a pointerenter event</title>
    <style>

div {
    position: absolute;
    top: 0;
    left: 0;
    width: 100px;
    height: 100px;
    background-color: black;
}

    </style>
    <script>

function logEvent(event)
{
    console.log(`${event.type}@${event.target.id}`);

    if (event.type == "pointerenter" && event.target.id === "top")
        event.target.innerText = "";
}

document.addEventListener("DOMContentLoaded", event => {
    for (let div of document.querySelectorAll("div")) {
        for (let eventType of ["mouseenter", "pointerenter", "mouseleave", "pointerleave"])
            div.addEventListener(eventType, logEvent);
    }
});

    </script>
</head>
<body>
<div id="top"><div id="middle"><div id="bottom"></div></div></div>  
</body>
</html>

If you move your mouse pointer over the black rectangle, which produces a "pointerevent" event that removes all of its content, the different browsers I've tested dispatch all different events.

Should the Pointer Events and/or UI Events specifications discuss what should happen in this situation?

Safari (ToT build)

pointerenter@top
mouseenter@top
pointerenter@middle
mouseenter@middle
pointerenter@bottom
mouseenter@bottom
pointerleave@bottom
mouseleave@bottom
pointerleave@middle
mouseleave@middle
pointerenter@top
mouseenter@top

Firefox (67.0.1)

pointerenter@top
pointerenter@middle
pointerenter@bottom
mouseenter@top
pointerleave@bottom
pointerleave@middle
pointerenter@top

Chrome (74.0.3729.169)

pointerenter@top
pointerenter@middle
pointerenter@bottom
pointerenter@top
mouseenter@top

NavidZ commented 5 years ago

Thanks for catching this. The only thing I found in UI event spec regarding this was: "If the event target (e.g. the target element) is removed from the DOM during the mouse events sequence, the remaining events of the sequence MUST NOT be fired on that element."

It certainly doesn't seem very clear and isn't very specific particularly in examples like what you gave. I do think that UI events spec is a better place to have this definition though. i.e. what happens to the propagation path when some of the nodes are changing during the event dispatching. Do we just keep firing them to the already calculated propagation path? @garykac

NavidZ commented 5 years ago

Talked about this issue in PEWG bi-weekly call and we decided to follow up on this issue with UI events folks first. Hopefully, we can define things better when @garykac procedural rewrite of the UI events spec is done. Let's follow up on this after/during TPAC.

patrickhlauke commented 4 years ago

Discussed on PEWG call today https://www.w3.org/2020/07/22-pointerevents-minutes.html#item01 - @smaug---- to catch up with @garykac about the fundamental/underlying vagueness (or not, depending on reading) of UI events spec for a decision there - then see based on that what spec here should do/reference.

garykac commented 4 years ago

I have an early draft for some proposed algorithms for UIEvents.

This document is still very much a WIP and it's in a separate document for now until it matures and has some general acceptance. But I feel that it's a reasonable start for documenting the UIEvent algorithms.

Of interest to Pointer Events, is the algorithm for handling mouse move events.

I added hooks where the pointer events would be called, and broke them out into a separate Pointer Event section. My intent is that the Pointer Events section of that document would eventually be moved into the Pointer Events spec.

Because of the lack of agreement between UAs on the firing order for the events, I needed to pick one that fit well with the algorithm for the other mouse events. Because each PointerEvent has a corresponding MouseEvent, the events are interleaved and the PointerEvent method is given the mouse event that is about to fire. This allows it to copy attributes if a PointerEvent needs to be created and fired.

I left most of the PointerEvent details as TODOs because I'm primarily interested in when/where to call the PE hooks rather than the exact details of how the PE spec decides if the PE should fire.

Let me know if you have any comments or feedback.

smaug---- commented 4 years ago

Thanks Gary. That algorithm looks quite reasonable to me. It seems to follow closer the model webkit has, and not the model blink and Gecko have (and even those have some differences.) Would blink be ok to change the behavior?

samuelmtimbo commented 4 years ago

I recently filled (what I found to be) a bug on Chromium particularly related to this discussion (which I just found out about). It might be a good use case to watch for:

"It is a common practice to listen to "pointerenter" and "pointerleave" events to keep track of which pointers are currently "visible" on the screen, which works well for any kind of pointer device (mouse and touch, for instance). When a "pointerleave" event is ignored as a result of removing the target element from the DOM (which might be a common operation), the system will end up holding a bad state, incorrectly signaling the pointer is still visible forever."

The analogy with Mouse Events gets trickier when dealing with a device that "does not support hover". I believe the Pointer Events spec should include this case and advise for firing "pointerleave" following up "pointerup" as it is already expected for that kind of device, although it is not currently what is happening on flagship mobile browsers (I tested on Chrome for Android and Firefox for Android). At the very least, browsers should consider firing "pointercancel" when the target element is removed (which I would not find intuitive, but still workable), otherwise I wouldn't know a reliable way of keeping track of such state.

I hope this is helpful! @garykac

patrickhlauke commented 4 years ago

@NavidZ @mustaqahmed any more thoughts on this issue?

smaug---- commented 4 years ago

Not related to this issue, but comment https://github.com/w3c/pointerevents/issues/285#issuecomment-725120778, why are pointerenter/leave used for tracking pointers? Those are significantly less performant than pointerover/out/move.

samuelmtimbo commented 4 years ago

@smaug---- the reason is that pointerover/out bubble so the container element will receive such events every time the pointer has entered/left a descendant, which would not make sense for the simple goal of tracking which pointers are currently visible on the container. This is particularly problematic (and less performant, depending on the handler logic) when the container has many children/grandchildren/etc. This page demo this behavior: https://glimmer-separated-spell.glitch.me/.

NavidZ commented 4 years ago

Sorry I missed the couple last comments. While I agree that the problem of dangling active pointers needs a better way of addressing as @samuelmtimbo mentioned, I need to mention the issue is more complicated when it comes to mouse.

There was a reason that we switched to decoupling mouse boundary events and pointerevents boundary events. There was this issue #279 that @graouts previously filed that references the original old issue regarding the moues compat events #35.

I suggest reading the comments on those issues before changing any current behavior but tl;dr is that there is no guarantee of one to one relation between pointerevents and mouse events (@garykac @smaug----). Pointerevents are coming each pointer stream and UA will generate compat mouse events based on all of those (whether the device is a mouse or a touch). Mouse events (whether generated by touch or mouse or both interacting with the screen at the same time) needs to be consistent by themselves to satisfy the pages that are only listening to mouse events. The same goes for pointerevents. So you can have a situation that you get a pointerenter event say from a touch (that UA decides to send a compat mouse event for) but there is no matching mouseenter as the mouse pointer was already there on that element (say because of the earlier mouse device moves).

patrickhlauke commented 3 years ago

@smaug---- @mustaqahmed @flackr when you get a chance, mind looking over this as well?

mustaqahmed commented 3 years ago

There are three separate questions here: what's the correct pointer event sequence before/after the node deletion? And what's the correct compat mouse event sequence here?

Here is a copy of the same repro reported above.

q1. Pointer Event sequence before deletion

I think the core problem here is that we don't know how deleted nodes should affect the repeated dispatch of mouseenter/mouseleave events. We are talking about the scenario mentioned below Figure 3 in UI Events Spec. The spec is silent about how the event dispatch works for deleted child nodes, and the silence is understandable because it is impossible to specify every possible corner cases. So any of the following interpretations are valid (and there can be more):

"Middle" and "bottom" shouldn't receive any events because they were deleted by the "top" event handler which was executed first. No browsers follow this.
"Middle" and "bottom" should receive all pointerenter and pointerleave events because the events were dispatched before "top" event handler started executing. All browsers follow this, so let's not worry about it for PE (but UIEvent spec needs work).

q2. Pointer Event sequence after deletion

The deleted nodes receive pointerleave events in Firefox and Safari, and not in Chrome. I think UI Events Spec doesn't say anything so we can ignore this too.

But all three browsers see an extra pointerenter at "top" after the deletion and that is clearly wrong: "top" shouldn't see two consecutive pointerenter without a pointerleave in-between.

q3. Compat mouse events

As Navid pointed out: we agreed that the compat mouse events should be fired in a way that makes the whole stream of compat events sensible in a multi-pointer scenario. If we have a correct pointer event sequence (last two points above), this should follow from #35 discussion.

garykac commented 3 years ago

I wrote up a proposed algorithm for event dispatch in this case:

https://w3c.github.io/uievents/event-algo.html#handle%20native%20mouse%20move

The draft algorithm matches the way browsers currently behave - the events are sent to the elements that have been removed. With regards to PointerEvents, the draft has hooks that call out to special PE algorithms for firing those events. I believe that the current set of hooks is sufficient, but until the PE algorithms are fleshed out we won't know for sure.

Suggestions for improving this algorithm are welcomed. The plan is to eventually fold this dispatching algorithm (with PE hooks) into the UI Events spec, so that the PE algorithms to be specified in the PE spec.

patrickhlauke commented 3 years ago

Marking this as future (not blocking v3) as the fundamental problem here is one for UI Events spec to tackle, with extra hooks for PE once it's done there.

pygy commented 7 months ago

pointerleave isn't fired on a parent element when a child is removed and the pointer is out of the parent's bound.

example (hover over the child, coming from below).

This is consistent among browsers, but surprising for tracking pointers and "hover" state.

mustaqahmed commented 7 months ago

pointerleave isn't fired on a parent element when a child is removed and the pointer is out of the parent's bound.

example (hover over the child, coming from below).

This is consistent among browsers, but surprising for tracking pointers and "hover" state.

This is working as intended. To track "hover" state, pointerenter and pointerleave are relevant only when event.eventPhase == Event.AT_TARGET.

EDIT: I meant pointerover and pointerout events instead!

pygy commented 7 months ago

This seems tautological for an event that doesn't bubble... I guess there's a bit I don't understand.

In this scenario, the child element is moving. If you

put the pointer on the path of the child and wait for it to overlap,
move the pointer while remaining within bounds
wait for the child to move away
then move again

You'll get the pointerleave for both the child and the parent at the fourth step.

https://github.com/w3c/pointerevents/assets/54515/5f46fec3-b6a0-42db-9c6f-03abeeb77541

In the child removal scenario example I suppose that the browser doesn't send the event to the child because it's been detached, but I don't get why the parent is not notified.

mustaqahmed commented 7 months ago

This seems tautological for an event that doesn't bubble...

Yikes, in my last post I meant to say non-bubbling pointerover and pointerout events to infer the "hover" state 😞.

In the child removal scenario example I suppose that the browser doesn't send the event to the child because it's been detached, but I don't get why the parent is not notified.

Yes, thanks, your repro is another way to look at the same problem we discussed above: in "q2" here we have an extra pointerenter because the hovering pointer causes the second pointerenter as if the first one is "forgotten", and your repro isolates the first one. According to the spec this seems working as intended because "the pointer didn't move off from an (existing) element" through the deletion. But the pairing mismatch question remains unresolved here unfortunately.

pygy commented 7 months ago

If it were to fire, pointerout wouldn't be reliable either because it bubbles and can be cancelled before reaching its target.

But pointerout doesn't fire on the child in Chrome.

Also, it doesn't necessarily fires on the parent if its boundary isn't crossed. The parent sees the event passing by while capturing/bubbling (or doesn't if the chld has been removed).

For pointerleave, in the example with the moving element, the pointer hasn't moved off the child either. In both cases the element has been moved away from the pointer, in one case out of the document.

It would be nice if the platform provided a reliable way to track what the pointer is hovering at any point given a dynamic document. That state is already being tracked by the browser for applying CSS.

Edit: Another problematic case is the reparenting a hovered element.

You get pointerenter/pointerover on the first parent, and pointerleave/pointerout on the second one only. Meanwhile CSS tracks it all without any problem.

flackr commented 7 months ago

In the child removal scenario example I suppose that the browser doesn't send the event to the child because it's been detached, but I don't get why the parent is not notified.

The new spec spec text should specify this as the previous target will track the still attached parent of the removed node

From pointer-events 4.1.3 Firing events using the PointerEvent interface

If the previous target at any point will no longer be connected [DOM], update the previous target to the nearest still connected [DOM] parent following the event path corresponding to dispatching events to the previous target, and set the needs over event flag to true.

Then on the next move, we specify that we treat the pointer as moving from this node per this text:

The user agent SHOULD treat the target as if the pointing device has moved over it from the previous target for the purpose of ensuring event ordering [UIEVENTS].

This should be implemented by the BoundaryEventDispatchTracksNodeRemoval feature in chromium.

pygy commented 7 months ago

What about the reparenting scenario ? As browsers work now the first parent isn't left, and the second isn't entered.

As an author, there are two kind of events I'd like to have access to:

one set that matches those that govern the CSS :hover state
one that reacts not only to pointer changes, but also to dynamic elements moving to the pointer (like the dragenter/over/leave do, but with the over hook that only fires when something has changed (dragover fires on a timer even if neither the pointer nor the element has moved, I suppose it has a purpose in the drag/drop scenario but this seems wasteful for a general handler).

In both cases, it would be ideal to get a guaranteed pointerleave guaranteed for each element that got a pointerenter when the element isn't in focus anymore.

garykac commented 4 months ago

I merged some draft algorithms for MouseEvents into the UI Events spec, along with some proposed hooks that should eventually be moved into PointerEvents.

Can someone take a look at the placement of the hooks and identify any other hooks that you might need (and also possibly propose better names).

w3c / pointerevents