Event improvements

[Korijn]

Based on previous experience with (pointer, mouse and keyboard) events in browsers, I think it would behoove the event specification and implementation to be expanded a little:

• Include pointer_id in pointer events. The current touches event property is useless as you can't tell from the event which pointer an event corresponds to (unless you do some rather involved bookkeeping).

• Include buttons in wheel events (added by AK).

• I have some questions as well:

• Why is there a double click event but no single click event?

• What defines a double click event exactly?

[almarklein]

Include pointer_id in pointer events.

That sound like a good idea!

Why is there a double click event but no single click event?

Mmm ... I'm not sure, tbh. I think my reasoning was that apps use pointer-down more often than click, while double-click can be used to implement specific functionality. I suppose adding a click event would make sense for completeness.

What defines a double click event exactly?

Whatever the system/browser defines it as - the implementation in jupyter_rfb uses the standard dblclick event.

[Korijn]

Include pointer_id in pointer events.

That sound like a good idea!

Great, then we can implement that!

I suppose adding a click event would make sense for completeness.

Well, the reason I bring this up is because I see some trouble with click events in jupyter-rfb. The value of a click event lies in its definition (emphasis mine):

An element receives a click event when a pointing device button (such as a mouse's primary mouse button) is both pressed and released while the pointer is located inside the element.

If the button is pressed on one element and the pointer is moved outside the element before the button is released, the event is fired on the most specific ancestor element that contained both elements.

This is a beautiful definition since it makes it easy for applications to differentiate between "drags" and "clicks".

However, applications which don't have access to click events, but only pointer down/move/up, still need to differentiate "clicks" and "drags". So they typically use something like the mouse movement between pointer down and up as a heuristic, but that doesn't always work as a user would expect it to (since it does not align with the definition above which is the standard behavior in the browser and in native desktop interfaces).

Getting to the point now, jupyter-rfb events (and wgpu canvas events) are always defined on a single element (the canvas). So you can't really implement the above definition. There is only one element, so every "drag" is also always a "click".

So I'm thinking, click, and double click, can be implemented in higher level libraries which do have an idea of what "elements" are represented in the canvas (such as pygfx and vispy), but not at this high level.

Therefore, I don't think click/double click really have a place in jupyter-rfb.

What do you think?

[Korijn]

Passing along the timestamp of an event would also be useful.

One example use case in order to detect click/double clicks, as explained above. :)

(Note to self: you can get the OS double click interval using pywin32. see example.)

[almarklein]

Good point about there always being one element from the canvas point of view. That, and because quite often there will be some sort of elements in the scene, makes the default click and dblclick useless indeed.

I see two options:

• The jupyter_rfb spec does not define click nor dblclick. This means an API breakage because we'll be removing the double-click.
• We implement click and dblclick, but using our own definition based on timings and absolute pixel movement only.

The advantage of the latter is that it's a reasonable solution that apps can use directly. Apps that want to have something based on their own definition of elements can easily ignore and re-define these events.

[berendkleinhaneveld]

Another thing: the spec for wheel events does not say anything about button or buttons. These values could come in handy though, just like the modifiers.

[almarklein]

Not sure why I did not spot this earlier, but the keys in the touches dict are pointer-id's. So if you handle something multi-touchy, you disregard the fields in the root of the pointer event, but use the touches to track the different moving pointers (its one of these touches that triggered the event).

In #78 I clarify this in the events spec. An also cover the other cases mentioned in this issue.