Wishlist: map sequence of events happening in a quite short timeframe into another event

[callegar]

Hi,

I have a laptop (Chuwi Hi10X) with a particularly weird keyboard/trackpad combo. The trackpad is actually reported as a mouse and tap/click actions on it are reported in a peculiar way (seen with evtest)

• Single finger tap -> LEFT BUTTON
• Double finger tap -> RIGHT BUTTON
• Triple finger tap -> Not reported by the mouse device, but by the keyboard device. You get: press event for LEFT_META, press event for key S, release event for LEFT_META, release event for S.

The rationale for this is beyond my understanding. Another annoying thing is that certain swipe actions on the trackpad are also reported as keyboard events. Specifically:

• Swipe from top to botton starting at a trackpad edge -> You get: press event for LEFT_META, press event for key D, release event for LEFT_META, release event for D.

Now, I would obviously like to remap these annoying key sequences. For instance, I would like the triple finger tap to result in a middle button click and the LEFT_META+D to be avoided altogether (because it triggers a show desktop action). It is my understanding that evsieve cannot be used for this purpose right now, as this would require to recognize the sequence of two key presses happening in a particular order and in a particularly restricted amount of time (so that a human generated, and as such more time-spaced LEFT_META+D press can be distinguished from the trackpad generated one). Is my understanding that evsieve cannot do what I would like to do right now correct?

I also understand that python-evdev would probably let me do what I would like to do (or maybe even evmapy that is based on python-evdev). However, I must confess that using python for processing keyboard events (on which I would like to have as little latency as possible) strikes me as a strange choice.

Would it be possible (and reasonable according to your priorities) to enhance evsieve to catch things like a sequence of keys pressed in order in a quite short timeframe, remove them from the event queue and trigger a configured input event in their place on some virtual input device? Or alternatively, is there already some tool to do so?

[KarsMulder]

Yes, you're right that there is currently no way in evsieve to conditionally block an event based on whether or not another event is sent in the near future.

I think evmapy isn't doesn't solve this problem either; while evmapy can cause a key to happen on a Meta+D sequence, from what I can tell evmapy cannot conditionally block that Meta key based on whether or not a D key will follow up shortly.

Would it be possible (and reasonable according to your priorities) to enhance evsieve to catch things like a sequence of keys pressed in order in a quite short timeframe, remove them from the event queue and trigger a configured input event in their place on some virtual input device?

I am undecided on the "sequence" part (i.e. whether the order should matter), but otherwise yes: a feature that delays events that can trigger a hook by a certain amount of time and drops those events if the hook is actually triggered within that amount of time, would be a great feature to have and shall be worked on.

However, I must confess that using python for processing keyboard events (on which I would like to have as little latency as possible) strikes me as a strange choice.

The latency of Python is lower than you'd expect. For a trivial scripts like "read events from one device and write them verbatim to another", the difference in latency between evsieve and an efficient Python implementation is about 0.05ms.

It's not zero, but even if you have a 240hz monitor, the difference in latency would be less than 1/80th of a frame. (The difference may become larger if you do complex event processing with several layers of abstraction in your Python script.)

In this context, "efficient Python implementation" means using the read_loop or select / selectors API for polling device events; using the asyncyio API will add another 0.2ms of latency.

[KarsMulder]

By the way, evtest should print a bunch of -------------- SYN_REPORT ------------ lines. Is there a SYN_REPORT line between the KEY_LEFTMETA and the KEY_S events when you do a triple finger tap?

[KarsMulder]

The main branch (commit 8633a27d right now) of on the git repository now has a rudimentary implementation* of three new features to enhance the --hook argument. A rough (incomplete) description of them:

• --hook withhold: if the withhold flag is specified, then all events that may trigger this hook are delayed until it is clear they will not trigger the hook. If the hook triggers, then the events are dropped.
• --hook period=SECONDS: all events that trigger this hook must be sent within a limited time period from each other, otherwise they won't count.
• --hook send-key=KEY: sends a key_down event for the specified key when this hook is triggered, and a key_up event once one of the keys that trigger this hook is released.

(*Rudimentary implementation: there are some design and implementation issues (aka. known bugs) left, but it might already work sufficiently well enough to solve your problem.)

Example script:

evsieve --input /dev/input/by-id/keyboard domain=kb grab persist=reopen \
        --input /dev/input/by-id/mouse    domain=ms grab persist=reopen \
        --hook key:leftmeta key:s withhold period=0.001 send-key=btn:middle@ms \
        --hook key:leftmeta key:d withhold period=0.002 \
        --output @kb \
        --output @ms

The first design issue that I didn't realise until I wrote the above script: the first hook withholds the leftmeta key from the second hook. In this case this can be worked around by giving the second hook a longer period, which is somewhat fine in this case because both periods are tiny anyway, but this composes badly for user-triggered hotkeys like trying to detect Ctrl+F1.

This feature goes back to the drawing table.

[sergiocallegari]

Hi, sorry for the delay and thanks for the help!

By the way, evtest should print a bunch of -------------- SYN_REPORT ------------ lines. Is there a SYN_REPORT line between the KEY_LEFTMETA and the KEY_S events when you do a triple finger tap?

No, they came together between a single SYN_REPORT couple of guards, and they do twice, first for the press and then for the relase, namely

Event: time 1641540694.761712, -------------- SYN_REPORT ------------
Event: time 1641540697.945755, type 4 (EV_MSC), code 4 (MSC_SCAN), value 700e3
Event: time 1641540697.945755, type 1 (EV_KEY), code 125 (KEY_LEFTMETA), value 1
Event: time 1641540697.945755, type 4 (EV_MSC), code 4 (MSC_SCAN), value 70016
Event: time 1641540697.945755, type 1 (EV_KEY), code 31 (KEY_S), value 1
Event: time 1641540697.945755, -------------- SYN_REPORT ------------
Event: time 1641540698.081821, type 4 (EV_MSC), code 4 (MSC_SCAN), value 700e3
Event: time 1641540698.081821, type 1 (EV_KEY), code 125 (KEY_LEFTMETA), value 0
Event: time 1641540698.081821, type 4 (EV_MSC), code 4 (MSC_SCAN), value 70016
Event: time 1641540698.081821, type 1 (EV_KEY), code 31 (KEY_S), value 0
Event: time 1641540698.081821, -------------- SYN_REPORT ------------
...

As soon as I recover from the Covid vaccination that is troubling me with days of headache I'll try your latest code...

[sergiocallegari]

... was way too curious... suffered from the headache but tested anyway... works!!!!

[sergiocallegari]

So, I have come down with this:

#! /bin/sh

evsieve --input \
        /dev/input/by-id/usb-HS-C109S-US-01-00-_USB_Keyboard-event-kbd \
        domain=kb grab persist=reopen \
        --input \
        /dev/input/by-id/usb-HS-C109S-US-01-00-_USB_Keyboard-if01-event-mouse \
        domain=ms grab persist=reopen \
        --hook key:leftmeta key:s withhold period=0.001 send-key=btn:middle@ms \
        --hook key:leftmeta key:d withhold period=0.001 \
        --hook key:leftmeta key:a withhold period=0.001 \
        --hook key:leftmeta key:b withhold period=0.001 \
        --hook key:leftmeta key:tab withhold period=0.001 \
        --output @kb \
        --output @ms

which includes the actual keyboard and mouse by-id names for the Chuwi hardware. It is a bit longer than I expected, because after some check I noticed that each touchpad edge causes a keyboard combination to be triggered. The one with the "A" I do not even know what stands for in Windows.

Now my system is very much improved in usability. I have a question. In your example the period values are different. Do they need to be so? I have tested always using 1ms and it seems fine.

Maybe there is still a marginal space for improvement, because when you slide and you start at an edge, at the beginning the sliding is unresponsive, and the reason why is obvious. Possibly what you should have is that at the release event for say leftmeta+d, you simulate some downwards mouse movement. I will experiment with that.

[KarsMulder]

Now my system is very much improved in usability. I have a question. In your example the period values are different. Do they need to be so? I have tested always using 1ms and it seems fine.

In the example where only two hooks were used, I believe it works fine because of implementation details. Theoretically, the first hook should release the key:leftmeta event at the same timestamp as when the second hook should release the key:d event, whether the Meta+D combination is caught depends on which of those two tasks is processed first. Because of the current implementation of the scheduler, the key:leftmeta event from the first hook is always released first, but I haven't made up a formal model for how the scheduler should behave yet and as such didn't want to recommend a script that relied on minute implementation details for correctness.

When multiple hooks are contending for the same event, it no longer works. For example, the following script will print Shift+B whenever you press the B key, but it will stop doing that if you change all the periods to 0.001s, because then the fourth hook releases the key:b event before the key:leftshift event reaches the hook.

evsieve --input /dev/input/by-id/keyboard grab \
        `# Use this map to simulate sending Shift+B at events the exact same time.` \
        --map key:b key:leftshift key:b \
        --hook key:leftshift key:s   withhold period=0.001 exec-shell="echo Shift+S" \
        --hook key:leftshift key:d   withhold period=0.002 exec-shell="echo Shift+D" \
        --hook key:leftshift key:a   withhold period=0.004 exec-shell="echo Shift+A" \
        --hook key:leftshift key:b   withhold period=0.008 exec-shell="echo Shift+B" \
        --hook key:leftshift key:tab withhold period=0.016 exec-shell="echo Shift+Tab" \
        --output

I am considering removing the withhold flag from the --hook and adding a --withhold argument instead, something like:

evsieve --input /dev/input/by-id/keyboard grab \
        `# Use this map to simulate sending Shift+B at events the exact same time.` \
        --map key:b key:leftshift key:b \
        --hook key:leftshift key:s   period=0.001 exec-shell="echo Shift+S" \
        --hook key:leftshift key:d   period=0.001 exec-shell="echo Shift+D" \
        --hook key:leftshift key:a   period=0.001 exec-shell="echo Shift+A" \
        --hook key:leftshift key:b   period=0.001 exec-shell="echo Shift+B" \
        --hook key:leftshift key:tab period=0.001 exec-shell="echo Shift+Tab" \
        --withhold \
        --output

Where --withhold delays all events that may trigger any of the hooks before it, and drops them if the hooks are actually triggered. This also allows for a nice interface if the user wants to withhold some keys but not others: --withhold key:leftshift could withhold the shift key if it could potentially trigger one of the hooks, but would not withhold the other keys.

I still have some design questions about the correct semantics and implementation of such an argument though (e.g. What if the user only wants to withhold events from some hooks instead of all of them? What happens to events that activated a --hook but then went through a --map before reaching the --withhold? What if a --map maps an unrelated event into an event that is identical to an event that triggered a --hook argument? What happens if a event that can activate hooks A and B gets followed up by an event with the same keycode that can activate hook B but not A, and then hook A subsequently triggers because of another event with the same keycode but different domain? Can I be certain that this argument won't cause stuck keys because it dropped the wrong events?)

[sergiocallegari]

I confirm that the current implementation is fragile when you need to deal with more than a couple of key combinations. Indeed, in my case I do not seem to succeed to reliably catch all the gestures and "swipe movements" from the touchpad edges. At the top I have the three finger tap and the swipe from top and they always work. But then I also have the swipe from left, swipe from right and swipe from bottom and they are not caught reliably. As a matter of fact, changing the period in an increasing fashion by 1ms does not seem to help that much.

I also have this problem that sometimes the desktop environment seems to stop recognizing the double tap with a single finger and I cannot say if this was always the case or has started now with the usage of evsieve or if it only happens when the system is loaded (this is a Celeron J with no active fans, so it is quite easy to load significantly).

The last thing that I would like to report is that rather often it is impossible to restart evsieve after it has been stopped (e.g. by ctrl+C or sigterm). When you try to start it for the second time it says that it cannot grab the mouse device and exits immediatly.

[KarsMulder]

As a matter of fact, changing the period in an increasing fashion by 1ms does not seem to help that much.

The period of each hook needs to be longer that the sum of the period of all hooks before it, therefore the period needs to be exponentially increasing (e.g. 0.001, 0.002, 0.004, 0.008, 0.016). A linearly increasing period does not work because 0.001 + 0.002 + 0.003 > 0.004: it takes 0.006 seconds for the key:leftctrl event to reach the fourth hook, but the fourth hook already releases the key:b event earlier at 0.004 seconds.

The last thing that I would like to report is that rather often it is impossible to restart evsieve after it has been stopped (e.g. by ctrl+C or sigterm). When you try to start it for the second time it says that it cannot grab the mouse device and exits immediatly.

Now this is a serious issue.

The most likely explanation is that an evsieve process is still running somehow: the kernel should automatically ungrab all devices that were grabbed by terminated processes, so even if evsieve were to be SIGKILL'ed and fail to do cleanup, the input devices should still end up ungrabbed. Try using ps aux | grep evsieve to check if evsieve is still running somewhere in the background.

If an evsieve process is still running somewhere and is ignoring SIGINT/SIGTERM, would you mind sharing a backtrace of that process? The gdb debugger can generate a backtrace the following way:

# Obtain the process ID with "ps aux | grep evsieve",
# or "pgrep evsieve" if only a single evsieve process is running.
sudo gdb -p EVSIEVE_PROCESS_ID -ex backtrace -batch

I also have this problem that sometimes the desktop environment seems to stop recognizing the double tap with a single finger and I cannot say if this was always the case or has started now with the usage of evsieve or if it only happens when the system is loaded (this is a Celeron J with no active fans, so it is quite easy to load significantly).

Combining this with the above issue of evsieve possibly not quitting, I wonder if evsieve is stuck in an infinite loop somewhere. Could you check the CPU usage of evsieve processes you're running? The ps tool can show the CPU usage the following way:

ps -p EVSIEVE_PROCESS_ID -o %cpu

[callegar]

The period of each hook needs to be longer that the sum of the period of all hooks before it

So now I have 1ms 1ms 2ms 4ms 8ms 16ms... This works. Hope not to find out there are more nasty leftwinkey combinations generated from the touchpad to catch because exponentials get tough rapidly :-)

An evsieve process is still running somehow... likely... I had a script starting and stopping evsieve and maybe it was missing something. Now the script is different and I don't seem to get the issue anymore.

As soon as you finalize the syntax, I'll try to wrap up everything with a systemd unit file.

[callegar]

Looks like I need some more assistance...

I have just realized that there are some more weird replies from the chuwi keyboard. For instance, there are some special keys controlled by a "FN" button, but they work differently from other laptops. In my main laptop I have two FN keys representing a big and a small sun. If I press FN together with them, I get the key:brighnessup and key:brightnessdown events. On the Chuwi laptop, I have a single FN key with a sun-like icon. If I press it together with FN, I get super_l (aka leftmeta) + i.

Now I would like to map this to brightness up. I use:

--hook key:leftmeta key:i withold period=0.032 send-key:brightnessup

and that works. So far so good. However, because there is a single brightness key, I would also like to map Shift+FN + this brightness key to brighnessdown ... and I do not succeed in making this work.

For instance:

--hook key:leftmeta key:leftshift key:i withold period=0.064 send-key:brightnessdown

does not work.

Any suggestion from yours?

Another issue is that the number of weird key responses from this laptop is so large that I am already at delaying stuff by almost 1/10th of a second with period=0.064. So I wonder if the alternate operation mode you mentioned in previous comments is expected to fix the exponential growth of delay periods.

Thanks!

[KarsMulder]

The reason that your approach does not work is because upon pressing Meta+I, the first hook consumes the Meta and the I keys, ensuring that they will never reach the second hook.

In the latest version of the main branch, the --hook argument no longer supports a withhold clause, and instead a --withhold argument has been added. One --withhold argument is supposed to follow the set of hooks that need to have their events potentially withheld. In most situations, you would write something like the following:

    --hook key:leftmeta               key:i period=0.032 send-key=key:brightnessup \
    --hook key:leftmeta key:leftshift key:i period=0.032 send-key=key:brightnessdown \
    --withhold

This does solve the problem of exponential growth since any amount of hooks can now use the same period without withholding keys from each other.

However, in this case, the above script the combination Meta+Shift+I would send both a brightnessup and a brightnessdown key, kinda defeating the purpose. It is less elegant, but maybe making the second map send key:leftshift+key:brightnessup to key:brightnessdown solves your issue?

    --hook key:leftmeta key:i period=0.032 send-key=key:brightnessup \
    --withhold \
    --hook key:leftshift key:brightnessup period=0.032 send-key=key:brightnessdown \
    --withhold