SP5 Pen input registers as cursor click&drag rather than touch input?

[benjamin051000]

In Windows 10, the stylus is treated as touch input. For example, it could be used to scroll through a webpage and worked the same way your finger did.

From what I'm seeing on Linux, it's registered as mouse input: Instead of scrolling down a webpage (for example), it highlights the text as if I clicked&dragged across the text.

Is this by design? Is there any way to change this behavior? Thanks for all you do!

[StollD]

This is because linux input / hid drivers (and by extension iptsd) usually expose pens as mouse pointers and not the same way as fingers.

It is theoretically changeable, and I want to look into it, just havent found the time yet.

[StollD]

So I have looked into this, but it seems there are some caveats. You wouldnt be able to use the side button or the eraser, and at least Xournal++ wouldnt be able to distinguish between the touchscreen and the stylus. So either you can write (but not drag) with the screen, or you can't write with the stylus.

The code is ready for being pushed but given these problems I dont think it makes much sense to add that option. Or is there someone that would use the stylus like that?

[benjamin051000]

Thank you for getting back to me about this!

Hm... Perhaps I miscommunicated my description of stylus behavior on Windows.

I want the experience to be similar to Windows, in the sense that on applications that do not support "drawing" (e.g., anything besides Xournal++) the stylus registers as touch input (it works the same way your finger does). In apps that do support drawing (Xournal++ and equivalents), it registers as however that app expects a stylus to work. This may not be entirely accurate, perhaps later I can do some more in-depth experiments and get back to you.

This would be a great addition because on apps like Firefox, it is much more common to scroll than to select text. If I remember correctly, pressing and holding the side button while swiping with the stylus may convert the stylus on Windows to a select-text mode in browsers, but normal swiping on the screen scrolls. This feature would add a way to interact with apps that is very comfortable to use.

Is there something about the way the environment works that doesn't allow this to happen? The goal would be for apps like Xournal++ to work the same way they do now, but change functionality across other apps.

I'm wondering if Windows treats stylus input as something entirely different than touch or mouse input, which allows for greater customization depending on your app... I don't know if this is related, but Windows does give the stylus its own cursor, which could be a hint that it's an entirely different input system than touch/mouse. Who knows though 🤷

[StollD]

Yeah, the issue is that it can only be a stylus or a touchscreen / finger. Nothing inbetween. Technically there is nothing in the linux input system that prevents a stylus from behaving like it does on windows, its simply that the popular DEs (GNOME and KDE) have decided to implement it just like a mouse pointer. It would be less of an issue if Xournal++ could differentiate between different touchscreens, because then you could just configure it to treat the stylus device as a stylus, but at least for me on KDE Wayland that is not the case. There isnt really anything that iptsd can do there, at least not easily.

[benjamin051000]

Thanks for your insight. Perhaps I'll discuss with the GNOME people about this. I know it's a pretty uncommon input method, but it's worth considering in future features.

[qzed]

its simply that the popular DEs (GNOME and KDE) have decided to implement it just like a mouse pointer

I don't think that's necessarily the problem with DEs / frameworks here. I'm not exactly sure how things work, so take the following with a grain of salt.

GTK and I assume QT as well do support stylus events. However, on GTK at least, those are pointer events. In essence (as @StollD mentioned) those are just like mouse events (or rather: mouse events are also sent as pointer events). So to differentiate, the application would have to check from which type of device the pointer motion comes. If it does not, everything just behaves like a mouse.

I think it might also be possible that if the application doesn't explicitly request touch events, those might also get translated to pointer events (although I know that GTK has explicit touch events). I guess the idea was that this would allow direct support of legacy applications since they don't need any special code to be usable with a stylus/touchscreen instead of a mouse.

So I think there may not be a good way to handle this in the DEs themselves without potentially breaking existing stuff.

[StollD]

Well, the idea was to make the stylus behave more like the touchscreen. That implies that the apps in question already understand touch events. If they were legacy apps, both the touchscreen and the stylus would result in a pointer.

And something in the stack must know that the device in question is a stylus, because as soon as I add BTN_STYLUS, BTN_TOOL_PEN or BTN_TOOL_RUBBER, the device becomes a pointer. If I omit those, it is treated like a touchscreen. Maybe this is also handled by libinput and not the DE?

[qzed]

Right, but then if you'd translate them to touch events by default, you'd break all the existing drawing apps that differentiate between touch and stylus and mouse/pointer. So you again need something that ideally the app controls, or some sort of list of drawing apps where you translate stylus events to pointers and everything else where you translate it to touch (which is pretty much unmaintainable).

So I don't see any feasible/reliable way to e.g. have scrolling by dragging the stylus without that being implemented in the app itself. Either by having the framework that provides the scrollable pane widget properly support stylus events for that or by having the app directly support it by handling the events itself.

But maybe I'm missing something. I guess DE/framework developers might be able to come up with something.