Cannot resize GTK3 applications using graphics tablet

[florentc]

• Version 1.5 on arch (official package)
• Wacom Intuos S tablet
• Debug log https://gist.github.com/florentc/3b4fb276c1a26c0d4c176bac34efd0d4
• Default configuration file

Description

Steps

• Launch terminal emulator (alacritty)
• Launch nautilus (or evince, pcmanfm, or other gtk3 apps) from terminal
• Use tablet pen to move cursor over nautilus window
• Hold MOD4 key down
• Hold right click button on tablet pen
• Move tablet pen

Expected result

The gtk3 window is resized according to cursor movement

Actual observation

It is not resized. The application registers a regular right click instead

Additional remarks

• same problem in floating mode (gtk3 apps do not resize when using the tablet pen)
• left click drag and drop with the tablet pen on the title bar of a gtk3 app in floating mode does not move the window around
• no problems when using regular mouse or trackpad
• no problem in other applications (e.g. when hovering the alacritty window instead of nautilus window in the above steps)

[Xyene]

For tablets;

• Resize functionality is bound to tapping down on a border, and dragging it. The state of the tablet buttons is not queried here.
• Move functionality similarly is tip down + mod held.

I think if right click + hovering works, it's by accident since it's going down the mouse emulation path when over pixel window borders. In truth, I wasn't aware you could resize windows like that with a mouse to begin with.

Resizing/moving containers by tablet input was implemented for tablet surfaces in https://github.com/swaywm/sway/pull/5421:

This PR "cheats" a bit since tool tips on container titlebars/edges actually go through pointer emulation, so we don't need to implement resizing logic in the tool tip seatop.

The key here, I think, is that GTK3 apps use client-side decorations, so Sway isn't the one that draws borders. Furthermore they also bind tablet handlers, so we don't go down the pointer emulation path either. That explains why things don't currently work; when I wrote the linked PR I assumed all borders wouldn't respond to tablet input, and that's possibly not the case for CSDs.

The fix for that would be to actually implement resizing in the tablet tool tip down seatop. At that point I think the tablet seatops will have (or nearly have?) feature parity with their pointer counterparts, so switching borders to accept tablet input might be a good idea.

I can probably take a look at this in the next week, unless someone else wants to.

Separately, I can see the use-case for allowing hover + tablet pen to trigger resizing the way right mouse button + drag does. My cheap tablet pen buttons are broken (in firmware) so I wouldn't be able to implement or test this, but I'd be happy to review a PR/provide pointers if you feel like implementing it.

[florentc]

Thank you your answer.

It is indeed possible to resize clients by dragging their borders around by keeping the pen down (or left button when using a mouse). I was not referring to this.

I was referring to resizing a client using mod4 held down + right mouse button held down. I think this is intended to work wherever the cursor is above the client to resize as long as mod4 is held down. When mod4 is held down, on i3 and on sway (using a mouse), right click events are captured by i3/sway to implement resizing and the client does not seem to register any right click event.

If I understood correctly, GTK3 apps manage their own decorations and event handlers for tablets. Thus, sway has no need to "convert" tablet events to fake regular mouse events because GTK3 apps can handle them natively.

However, when pressing mod4 + rightclickOnPenTablet, it seems that sway should handle that as an event targeted for sway itself (which should result in overriding the size of the client whether it is a GTK3 app or not) instead of considering the pen button event is targeted at the client underneath. Much like when a keyboard shortcut such as mod4+e does not send the "e" keypress to the active client. Or when using a mouse mod4 + right click does not relay the mouse event to the client.

Please forgive me if I got it all wrong. In any case I would be happy to assist in testing changes. Thank you very much for offering help to guide me around. I tried to debug a tablet bug a few month ago but have to admit I had a hard time getting the big picture of event handling for pointing devices.

[Xyene]

It is indeed possible to resize clients by dragging their borders around by keeping the pen down (or left button when using a mouse). I was not referring to this.

Yes, except it doesn't work for applications that both have CSDs and bind tablet handlers. That is a bug, even if it was not the one you were initially describing :)

If I understood correctly, GTK3 apps manage their own decorations and event handlers for tablets. Thus, sway has no need to "convert" tablet events to fake regular mouse events because GTK3 apps can handle them natively.

Correct. This is only important because for non-CSD apps, the pixel borders are treated as a "fake", pointer-only region, so pressing the pen down on them generates a mouse button right event, and the resizing works under pointer emulation. This doesn't work for CSDs if the application also binds the tablet events; we just don't start resizing because there's no code there.

However, when pressing mod4 + rightclickOnPenTablet, it seems that sway should handle that as an event targeted for sway itself (which should result in overriding the size of the client whether it is a GTK3 app or not) instead of considering the pen button event is targeted at the client underneath.

I agree it would be nice if this worked. The thing is there's no such thing as a "right button" (BTN_RIGHT) on a pen (though it may be convenient to refer to it as such). There's button 1 (BTN_STYLUS), button 2 (BTN_STYLUS2), etc. Applications that bind tablet handlers receive events that way. For those that don't, we always send an emulated right mouse button event regardless of which pen button was pressed.

https://github.com/swaywm/sway/blob/36c3c222d2418b30fea02c28ea21652ec90238c4/sway/input/cursor.c#L762-L772

We could pick one of those buttons and treat it as the resize button, but I remember reading that their numerical order doesn't necessarily correspond to their physical order, so the right choice for some pens might be different than for others.

When mod4 is held down, on i3 and on sway (using a mouse) ...

I'm pretty sure i3 has no (and needs no) explicit tablet support, and sees the input device as a fake mouse since everything is handled a level above it. A quick grep through i3 for tablet brings 0 results.

[florentc]

The thing is there's no such thing as a "right button" (BTN_RIGHT) on a pen (though it may be convenient to refer to it as such). There's button 1 (BTN_STYLUS), button 2 (BTN_STYLUS2), etc.

Oh I see! This is what I was missing. Thank you very much. This indeed explains a lot of what I have been experiencing in Wayland compositors, including sway.

We could pick one of those buttons and treat it as the resize button, but I remember reading that their numerical order doesn't necessarily correspond to their physical order, so the right choice for some pens might be different than for others.

You are right. I have already noticed inconsistencies between which button i3 arbitrarily chooses as right click button and which button gnome applications (such as nautilus) use to trigger context menus in wayland compositors. I have even witnessed differences between clients in the same sway session (although I think it might have involved apps through XWayland such as chromium).

This means tablet users have to trust all their apps to comply to the same conventions when it comes to handling tablet events.

There are other weird behaviors arising from that: for example some GTK3 applications such as pcmanfm fail to spawn context menus where the cursor is unless triggered by a "true" right click from a mouse. In some apps context menus behave differently on purpose. Instead of clicking on the desired element you have to hold the pseudo-right-button on the tablet pen down, and then release it once you have selected the desired element.

This results in a cumbersome experience with unpredictable behaviors when using a tablet as the main pointing device and a mix of applications including legacy X11 apps, GTK3 gnome apps, GTK3 third party apps, and so on.

I get that most of these are not sway bugs at all. Do you think introducing an optional setting to force mouse emulation is philosophically acceptable in the sway project? Some kind of forced mouse emulation that still provides pressure sensitivity like in i3.

[emersion]

Do you think introducing an optional setting to force mouse emulation is philosophically acceptable in the sway project?

I'd rather not add workarounds for clients in Sway. The better option is to fix clients.

[florentc]

This makes sense I understand. Instead of full mouse emulation, would an option to bind a tablet/pen button to free resize mode using mod4 (so that it is possible to resize without having to pin point borders or use a regular mouse / keyboard) be OK?

Thank you both for the clarifications.

Although my user experience with the tablet is not smooth enough for now so I can fully migrate to sway, I will try to experiment a bit more and compare behaviors to other wayland compositors in order to identify which quirks might be sway-specific and which result from decoupling of mouse and tablet in the Wayland protocol.

[Xyene]

would an option to bind a tablet/pen button to free resize mode using mod4 (so that it is possible to resize without having to pin point borders or use a regular mouse / keyboard) be OK?

I'd personally be in favor of this; trying to pinpoint 1px-wide borders with a high-sensitivity tablet is pretty annoying. Most of the work would have to be done anyway in order to implement CSD resizing behavior.