linebender / druid

A data-first Rust-native UI design toolkit.
https://linebender.org/druid/
Apache License 2.0
9.56k stars 569 forks source link

Mouse behavior status report #828

Open xStrom opened 4 years ago

xStrom commented 4 years ago

Background

This survey got started because of #821, which is related to #558 / #541.

I tested behavior on Windows 7 SP1, macOS Catalina, Ubuntu 19.04. The testing was done with druid master 3ffe5e4 (macOS / Linux) and #821 (Windows).

Data

Topmost = The druid window has the highest z-order and no other window no matter where they are located will occlude the druid window. Occlusion = When another window has a higher z-order than druid and is blocking the view to parts of the druid window. ⚠️ = Most pressing issues. ⌛ = It happens but not instantly.

HotChange(true) when window is gaining topmost status Windows macOS Linux/GTK
Cursor inside the window Yes✔️ No No
Cursor inside the window and then moves Yes✔️ Yes✔️ Yes✔️
HotChange(true) when window is gaining topmost status after clicking and holding the mouse button in another application Windows macOS Linux/GTK
Cursor inside the window No No No
Cursor inside the window and then releases hold Yes✔️⌛ No No
Cursor inside the window and then releases hold + moves Yes✔️ Yes✔️ Yes✔️
Window is topmost Windows macOS Linux/GTK
Receiving mouse events inside the window Yes✔️ Yes✔️ Yes✔️
Receiving mouse events outside the window No Yes✔️ No
Window is topmost + clicking and holding the mouse button in druid window Windows macOS Linux/GTK
Receiving mouse events inside the window Yes✔️ Yes✔️ Yes✔️
Receiving mouse events outside the window Yes✔️ Yes✔️ Yes✔️
HotChange(false) when window is losing topmost status Windows macOS Linux/GTK
Cursor inside the window No No No
Cursor inside the window and then moves No No No
Cursor inside the window but occluded Yes✔️ No No
Cursor inside the window but occluded and then moves Yes✔️ No No
HotChange(false) when window is losing topmost status after clicking and holding the mouse button in druid window Windows macOS Linux/GTK
Cursor inside the window No No No
Cursor inside the window and then releasing hold No No No
Cursor inside the window and then releasing hold + moving No No No
Cursor inside the window but occluded No No No
Cursor inside the window but occluded and then releasing hold Yes✔️⌛ No No
Cursor inside the window but occluded and then releasing hold + moving Yes✔️ No No
Window is not topmost Windows macOS Linux/GTK
Receiving mouse events inside the window Yes✔️ No Yes✔️
Receiving mouse events inside the window but occluded No No No
Receiving mouse events outside the window No No No
HotChange(false) when cursor moves to occluded part of the window Yes✔️ No Yes✔️
HotChange(false) when cursor leaves the window via an occluded part Yes✔️ No⚠️ Yes✔️
HotChange(false) when cursor leaves the window via a non-occluded part Yes✔️ No⚠️ Yes✔️
Window is not topmost + clicked and held the mouse button in druid window before it lost topmost status Windows macOS Linux/GTK
Receiving mouse events inside the window Yes✔️ Yes✔️ Yes✔️
Receiving mouse events inside the window but occluded No⚠️ Yes✔️ Yes✔️
Receiving mouse events outside the window No⚠️ Yes✔️ Yes✔️
HotChange(false) when cursor moves to occluded part of the window No No No
HotChange(false) when cursor moves to occluded part of the window and then releasing hold Yes✔️⌛ No No
HotChange(false) when cursor moves to occluded part of the window and then releasing hold + moving Yes✔️ No No
HotChange(false) when cursor leaves the window via an occluded part No Yes✔️ Yes✔️
HotChange(false) when cursor leaves the window via an occluded part and then releasing hold Yes✔️⌛ Yes✔️ Yes✔️
HotChange(false) when cursor leaves the window via an occluded part and then releasing hold + moving Yes✔️ Yes✔️ Yes✔️
HotChange(false) when cursor leaves the window via a non-occluded part Yes✔️ Yes✔️ Yes✔️
HotChange(false) when cursor leaves the window via a non-occluded part and then releasing hold Yes✔️ Yes✔️ Yes✔️
HotChange(false) when cursor leaves the window via a non-occluded part and then releasing hold + moving Yes✔️ Yes✔️ Yes✔️

Discussion

A bunch of these differences don't necessarily matter and will just give the druid app an OS-feel. Things like macOS not being able to show hover effects when the window isn't topmost, or Windows not showing hover effects for elements that are being occluded by other windows. However some of the differences here actually pinpoint more significant issues.

Windows

When we are capturing the mouse after MouseDown and the window loses topmost status or loses its capture state for any other reason, then we won't get a MouseUp unless it happens over a non-occluded part of our window. Maybe it's possible to re-capture somehow? If not, then there are two ways to solve this. One is to do nothing and document that not every MouseDown has a companion MouseUp and vice versa. An alternative solution would be to add a MouseCancel event which druid will synthesize whenever a MouseDown event arrives before the previous MouseDown received its MouseUp.

Thus it's educating to handle MouseCancel or educating that MouseUp and MouseDown are independent calls and can be dropped.

This issue also touches on what I mentioned in #396 where MouseUp was being eaten my modal dialogs. I hope to get back to that modal issue testing in a few weeks.

macOS

The main issue is if the window is hot and then it loses its topmost status, e.g Command+Tab. The window won't receive any more events but the hot status doesn't get reset. Maybe there's an event for when the window loses its topmost status? While handling that event could check if the mouse is captured (MouseDown + hold) and if not, then reset all hot states as we won't be receiving any more mouse events.

Linux/GTK

Linux receives mouse events even when not topmost, however the hot state can remain stuck when leaving the window area via and occluded area. Hopefully there's a way to handle this.

cmyr commented 4 years ago

Just saw this, excellent stuff, thanks!