iptsd-show magically fixes KDE touchscreen gestures

[dionisis2014]

This is a weird one. On my SP8, with KDE wayland, I have enabled touchscreen gestures (basically you can swipe from screen edges to trigger actions such as show all windows, desktop etc). Here is the problem: When doing a cold boot and trying to activate the gestures, its VERY hard to do so (maybe 1 out of 30 tries). I wanted to see if the touch didn't register and run sudo iptsd-show $(iptsd-find-hidraw). Everything looked normal and now, the gestures work reliably. And this is reproducible (on my machine at least). Every time I run the command, it magically fixes the issue. How? Why?

Environment

• Hardware model: Surface Pro 8
• Kernel version: Linux aeolus 6.3.5-arch1-1-surface #1 SMP PREEMPT_DYNAMIC Fri, 02 Jun 2023 09:46:51 +0000 x86_64 GNU/Linux
• Distribution: Arch Linux

`dmesg` output

[pastebin link](https://pastebin.com/v1TzFzDQ) password: G53mTvYDmb

[StollD]

Does this already happen while iptsd-show is running or does it only set it after you closed it?

Closing iptsd-show would reset the hardware back into singletouch mode, like if iptsd wasn't running. That could explain the different behaviour. Otherwise it would just be two apps reading from the same source.

[dionisis2014]

Sorry for the late reply. Its really weird if it resets after, because the gestures don't work before and during iptsd-show, but work afterwords. Now while running iptsd-show its in fullscreen and gestures are disabled. Alt-tabbing while and trying also doesn't trigger gestures, but that might be because a fullscreen app is running in the background (might be a KDE thing).

[StollD]

No, the reason is simply that while it is running, iptsd is still in charge of producing touch events. Closing iptsd-show resets the hardware into singletouch mode, which effectively disables iptsd. In that mode, the singletouch events come from the hardware directly.

The actual reason is that the events that iptsd produces might be unstable and jitter more than the hardware generated events, so that KDE doesnt properly register the gesture. I am trying to improve this for the next release, guess I need to add KDE gestures to the list of things to check.

[farline99]

KDE gestures from the edges of the screen are broken for a long time. SP6. When you move your finger from the black frame to the screen, the touchscreen does not read the first millimeter of your movement, which KDE gestures rely on -> they do not work. If this millimeter near the black frame was taken into account, then everything would be fine with gestures 😉

[plover43]

@farline99, Interesting that you mention the edges of your screen not detecting input. I'm experiencing the same thing on the bottom of my screen on my sp9 kde fedora38. For me it's more like the bottom 5mm aren't recognized.

When clicking the icons in the picture, I have to purposefully click pretty much just above them or no input will be registered.

[farline99]

Yes, if the panel is small, then the lack of reaction to clicking on the edges of the screen makes it very difficult to call widgets on it 🙂

[dionisis2014]

@StollD (I am sorry if I didn't understand correctly.) As I said, the gestures work after closing iptsd-show. They didn't before or during (as you explained), but work surprisingly consistently afterwwards. Is the hwardware reset from iptsd doing something extra that isn't done at startup?

[StollD]

IPTS has two modes: singletouch and multitouch. In singletouch mode, the hardware directly produces inputs for a single finger, e.g. the driver receives coordinates and forwards them to the input subsystem.

In multitouch mode the hardware does not produce coordinates, instead, it produces a capacitive heatmap showing the electrical resistance on the screen. This heatmap needs to be processed into coordinates, which is what iptsd does.

By stopping iptsd-show you are disabling iptsd and its processing and switch to the hardware based processing, which only supports a single finger and no pen. Since the processing algorithms are different, the generated inputs will also be different.

[dionisis2014]

Ok I am a little confuzed here. How are general multitouch gestures working in e.g. firefox (pinch to zoom etc) if IPTSD is disabled? What I observe is that BEFORE running iptsd-show gestures don't work, but do AFTER closing it. In both scenarios IPTSD is not running correct?

[StollD]

That is indeed weird. Does pinch to zoom work without launching iptsd-show? Or does that have the same issues as the KDE gestures?

Could you maybe post the output of systemctl status $(iptsd-find-service) after you closed iptsd-show and verified that pinch to zoom still works?

[dionisis2014]

It keeps getting weirder: BEFORE:

• [ ] Gestures don't work
• [x] Multitouch works

AFTER:

• [x] Gestures work
• [ ] No multitouch :(

My guess (with only what you told me) is that the 'reset state' after is somehow different from the one the hardware is initialized with. That, or something doesn't start up again after.

The output below is the same before and after running iptsd-show:

`systemctl status $(iptsd-find-service)` output

● iptsd@dev-hidraw0.service - Intel Precise Touch & Stylus Daemon
     Loaded: loaded (/usr/lib/systemd/system/iptsd@.service; static)
     Active: active (running) since Sat 2023-06-10 21:08:16 EEST; 2min 13s left
       Docs: https://github.com/linux-surface/iptsd
   Main PID: 373 (iptsd)
      Tasks: 1 (limit: 9314)
     Memory: 996.0K
        CPU: 2ms
     CGroup: /system.slice/system-iptsd.slice/iptsd@dev-hidraw0.service
             └─373 /usr/bin/iptsd /dev/hidraw0

Ιουν 10 21:08:16 aeolus systemd[1]: Started Intel Precise Touch & Stylus Daemon.
Ιουν 10 21:08:16 aeolus iptsd[373]: [21:08:16.531] [info] Metadata:
Ιουν 10 21:08:16 aeolus iptsd[373]: [21:08:16.532] [info] rows=46, columns=68
Ιουν 10 21:08:16 aeolus iptsd[373]: [21:08:16.532] [info] width=27389, height=18259
Ιουν 10 21:08:16 aeolus iptsd[373]: [21:08:16.532] [info] transform=[408.79105,0,0,0,405.75555,0]
Ιουν 10 21:08:16 aeolus iptsd[373]: [21:08:16.532] [info] unknown=1, [178,182,180,1,178,182,180,1,90,171,100,20,172,177,175,2]
Ιουν 10 21:08:16 aeolus iptsd[373]: [21:08:16.532] [info] Connected to device 045E:0C37

[StollD]

I assume by gestures you just mean the KDE gestures where you swipe in from the side of the display with one finger? In that case that would confirm my theory: You reset to singletouch mode, which does a different processing which makes the KDE gestures work, but because only a single finger is handled, you cannot pinch to zoom anymore.

[farline99]

Yeah, we are talking about this gestures from side of the screen.

[dionisis2014]

Is this an IPTSD issue then? If the hardware is in multitouch mode before why don't gestures work? Is the processing so different for one finger between single and multi touch mode?

[StollD]

Yes, this is an iptsd issue. And I cannot tell you the difference between the two processing algorithms, because the one for singletouch is proprietary and baked into the firmware. For iptsd we had to come up with our own one, which is still getting refined a lot.

[dionisis2014]

So the "working" one is from the firmware correct? Enabling 'show touch inputs' under desktop effects shows correct position and detection both before and after iptsd-show. I know KDE fixed a bug a long time ago that took into account sub-grid movements and made it impossible to trigger gestures unless you had a ruler. If there is anything I can do on my machine feel free to ask me. Thank you.

[StollD]

I believe I found the issue and the fix: Our processing algorithm is a bit too "precise": It can anticipate contacts being outside of the screen, so it returns negative coordinates. This confuses KDE. A version of iptsd that ignores these contacts makes the gestures work much more reliable, at least when you move the finger slowly.

I will try to experiment a bit more to make this work better when the finger moves more quickly.

[farline99]

It is awesome. Thank you for your work! ❤️

[dionisis2014]

I finally found the commits I had seen from KDE way back. This commit made it so sub-pixels are ignored and this one made it use an accumulative method to determine direction. This is the current way KDE handles gestures. As you can see it takes the absolute of the delta value and then a minimum of that (divided by some other stuff) and 1.0 . So a negative coordinate confusing it is unexpected.

[StollD]

https://github.com/KDE/kwin/blob/master/src/gestures.cpp#L153

If minimumX is 0 a negative X coordinate would fail this check.

[dionisis2014]

I am sorry. Indeed it is 0: https://github.com/KDE/kwin/blob/master/src/gestures.h#L112 . Should I file a bug report on KDE and close this one?

[StollD]

No, its an iptsd issue. Linux just happens to not complain about the negative values, even though they are outside of the allowed range.

[StollD]

I have made some changes to iptsd that make the KDE gestures work much more reliably. Contacts outside of the screen are now ignored, but if the center of the contact is close enough to the edge of the screen, the coordinates get clamped to the allowed range. So instead of e.g. -10 the value becomes 0.

If you want to try these changes out, you can download a development build of iptsd here: https://github.com/linux-surface/iptsd/actions/runs/5299078719. Scroll down to the list of artifacts and download <your distro>-latest. You will get a ZIP file with a package inside that you can install using your package manager.

[plover43]

This completely fixes the touchscreen edge gestures on my end, thanks so much for the fix

6.3.7-1.surface.fc38.x86_64 KDE Plasma Version: 5.27.5 KDE Frameworks Version: 5.107.0 Qt Version: 5.15.9

[farline99]

SP6. F38. Thank you. With latest patch gestures work as expexted on KDE ❤️❤️❤️

[dionisis2014]

Yeap. Works on me too. You are amazing! Should I close this issue or leave it until its in a release?

[StollD]

Lets close this now.

FYI, if the gestures still dont work well, or if you want to experiment a bit, there is a new config option in /etc/iptsd.conf called Overshoot, which controls how far outside of the display a contact can be to still get registered.

[plover43]

Timeline on getting this released?

[farline99]

Yeah, new iptsd release would be cool!