Trackpads not properly supported (Cmd-Click support)

[joshgoebel]

Ref: #16

Trackpad devices do not work for Cmd-click (or they would, but we don't pass on the tracking events so the trackpad doesn't allow any movement, which makes it a bit worthless).

[joshgoebel]

Well, I can't seem to get a Trackpad working no matter what I try... I tried just copied the raw set of numbers over: (from the capabilities list of my trackpad)

ecodes.EV_ABS: set([0,1,24,47,48,49,52,53,54,57,58])

I tried set(range(0,0x3f))... I tried printing the capabilities and them passing them in directly:

        3: [(0, AbsInfo(value=3879, min=-3678, max=3934, fuzz=0, flat=0, resolution=47)), (1, AbsInfo(value=2501, min=-2478, max=2587, fuzz=0, flat=0, resolution=44)), (24, AbsInfo(value=0, min=0, max=253, fuzz=0, flat=0, resolution=0)), (47, AbsInfo(value=10, min=0, max=15, fuzz=0, flat=0, resolution=0)), (48, AbsInfo(value=0, min=0, max=1020, fuzz=4, flat=0, resolution=0)), (49, AbsInfo(value=0, min=0, max=1020, fuzz=4, flat=0, resolution=0)), (52, AbsInfo(value=0, min=-3, max=4, fuzz=0, flat=0, resolution=0)), (53, AbsInfo(value=0, min=-3678, max=3934, fuzz=0, flat=0, resolution=47)), (54, AbsInfo(value=0, min=-2478, max=2587, fuzz=0, flat=0, resolution=44)), (57, AbsInfo(value=0, min=0, max=65535, fuzz=0, flat=0, resolution=0)), (58, AbsInfo(value=0, min=0, max=253, fuzz=0, flat=0, resolution=0))]

Once I add more than 1 or 2 things to EV_ABS all my output stops working completely - and all the inputs SEEM dead (they are still being read just fine) since they are being grabbed - and I have to use the emergency shutdown to get access to the system again.

[joshgoebel]

I even tried being JUST a trackpad:

    return UInput(name="Keyszer VIRTUAL Keyboard",
                  events={ 

        3: [(0, AbsInfo(value=3879, min=-3678, max=3934, fuzz=0, flat=0, resolution=47)), (1, AbsInfo(value=2501, min=-2478, max=2587, fuzz=0, flat=0, resolution=44)), (24, AbsInfo(value=0, min=0, max=253, fuzz=0, flat=0, resolution=0)), (47, AbsInfo(value=10, min=0, max=15, fuzz=0, flat=0, resolution=0)), (48, AbsInfo(value=0, min=0, max=1020, fuzz=4, flat=0, resolution=0)), (49, AbsInfo(value=0, min=0, max=1020, fuzz=4, flat=0, resolution=0)), (52, AbsInfo(value=0, min=-3, max=4, fuzz=0, flat=0, resolution=0)), (53, AbsInfo(value=0, min=-3678, max=3934, fuzz=0, flat=0, resolution=47)), (54, AbsInfo(value=0, min=-2478, max=2587, fuzz=0, flat=0, resolution=44)), (57, AbsInfo(value=0, min=0, max=65535, fuzz=0, flat=0, resolution=0)), (58, AbsInfo(value=0, min=0, max=253, fuzz=0, flat=0, resolution=0))],

        ecodes.EV_KEY: touchpad_btns,

        })

To see if i could just get touchpad data to pass thru, but no, that doesn't work either. Note that evdev shows ALL the data coming thru fine... if I connect all 3 it's just a large spiggot with all the events.... kb, mouse, trackpad, etc.

[joshgoebel]

Ok, got it trackpads are TRICKY... if you copy the input properties & vendor/product/version/bustype over then it works (as a trackpad) - you can't pass thru keyboard or mouse events, even if you declare them as supported events...

I'd say libiinput is using something entirely different for touchpads... you can be a mouse/keyboard or a touchpad, but NOT both...

[joshgoebel]

Ok, I think the right way to do this is to have event streams we modify and event streams we watch only... confirming we are a keymapper - not a mouse and trackpad mapper... so we modify keyboard steams, but for modifer-clicking we could watch the keyboard and trackpad streams, and when we see a click-like event we could use those to trigger waking of suspended keys in the hope that they have some type of interaction with the mouse/trackpad.

[joshgoebel]

Ugh, but that can't work because I think the button event would be on the stream before we got the modifier events on the stream - since we're doing it in reaction to the button event... and if we aren't generating the button event ourselves we have no way to push the modifiers down first.

☹️

[joshgoebel]

And worse the stream is ambiguous... is a trackpad in "tap to click" or "click to click" mode? The event stream for these two is entirely different (click to click sends a real BTN_LEFT) and I'm not sure it'd even be possible to figure out "tap to click" without a ton of detailed analysis... it doesn't send a discrete event... it just sends tons of touch and movement events - and the driver has to make sense of them.

So this is easy to solve for mice, but perhaps impossible to solve for trackpads?

[joshgoebel]

@redbearak I'm not sure trackpad CMD-clicking is a solvable problem... 😕

[RedBearAK]

Yikes. Had no idea it was going to be this complicated.

And the only thing that makes it work in Kinto is that there's no suspension delay on the modifier(s) until it(they) is(are) part of a full shortcut, right? So it "just works" and always has. It just also opens menus sometimes when you don't want it to.

As I realized earlier I also Shift+Click a lot, or even Alt+click in VSCode to get secondary cursors. I try to use the keyboard as much as possible but I'm no Wunderkind shuffling around in Emacs and a tiling window manager like a piano virtuoso. Most users will do some kind of Modifier+click action from time to time. There has to be a solution somewhere.

If a driver has to make sense of the touchpad tap-to-click then there has to be a way to listen to the driver when it sends the "click" event to the application. Sounds like this method is just tapping into things too early in the process.

[joshgoebel]

And the only thing that makes it work in Kinto is that there's no suspension delay on the modifier(s) until it(they) is(are) part of a full shortcut, right?

Right, it just smashes keys willy-nilly and then has to un-press them when it finds out the actual combo they belong to.

Most users will do some kind of Modifier+click action from time to time.

Oh sure... I'm not saying otherwise, just damn this is hard problem.

there has to be a way to listen to the driver when it sends the "click" event to the application

Well, proxy, not just listen. Listen is no good because we have to press the modifiers BEFORE the click... not after... the whole keymapper thing works because we grab the keyboard (preventing anyone from seeing it) and then output it ourselves - we sit between. So if you wanted to hack into X events at a higher level you'd need the same ability - to sit between/change events, etc...

[tieugene]

I don't know about Cmd-Click, but Shift-Click (select text from text cursor up to trackpad cursor) not works too. To be precise an hour ago (with 20220715 sources) this worked sometimes. Now (20220716 sources) this not works at all.

[RedBearAK]

@tieugene

I don't know about Cmd-Click, but Shift-Click (select text from text cursor up to trackpad cursor) not works too. To be precise an hour ago (with 20220715 sources) this worked sometimes. Now (20220716 sources) this not works at all.

Does it work if you hold the modifier key for at least one second before clicking? If so you just need to adjust the timeout length.

[tieugene]

Does it work if you hold the modifier key for at least one second before clicking? If so you just need to adjust the timeout length.

Yes, this works, thank you.

[RedBearAK]

Does it work if you hold the modifier key for at least one second before clicking? If so you just need to adjust the timeout length.

Yes, this works, thank you.

So, what you want to do is put something like this in your config file, probably somewhere near the top.

# Keyszer-specific config settings
# dump_diagnostics_key(Key.F7)
# emergency_eject_key(Key.F8)
timeouts(
    multipurpose = 1,
    suspend = 0.1,
)

You can leave the dump_diagnostics_key and emergency_eject_key lines disabled unless you want to customize the keys from the defaults of F15/F16.

Tune the suspend variable value to whatever time seems to work best for you, between 0.1 and 1 second. Leaving it very short like this has the side effect that the menu bar will sometimes activate in some apps when you press the Alt modifier (physical Super/Win under Kinto's remapping). You might try 0.3, but I found that to still be a bit too long at times.

Like any other change to the config file, you have to restart keyszer after saving the file.

[laech]

Would this https://github.com/mooz/xkeysnail/pull/137 work better instead of suspending modifiers? That's what I used to fix the Firefox-Alt issue and the likes.

[RedBearAK]

@joshgoebel

I just made a slight modification to output.py that seems to have dramatically improved the Mod+click (or Mod+tap) situation with my laptop touchpad. But I'm not going to try to put it in a PR because this is out of my wheelhouse and I don't have any idea yet whether this would somehow be harmful to users with other devices.

# Remove all buttons so udev doesn't think keyszer is a joystick
_KEYBOARD_KEYS = ecodes.keys.keys() - ecodes.BTN

# But we want mouse buttons, so let's enumerate those and add them
# back into the set of buttons we'll watch and use
_MOUSE_BUTTONS = {
    256: ["BTN_0", "BTN_MISC"],
    257: "BTN_1",
    258: "BTN_2",
    259: "BTN_3",
    260: "BTN_4",
    261: "BTN_5",
    262: "BTN_6",
    263: "BTN_7",
    264: "BTN_8",
    265: "BTN_9",
    272: ["BTN_LEFT", "BTN_MOUSE"],
    274: "BTN_MIDDLE",
    273: "BTN_RIGHT",

    330: ["BTN_TOOL_FINGER", "BTN_TOUCHPAD"],  # touchpad click
    333: "BTN_TOOL_DOUBLETAP",  # touchpad double tap
    334: "BTN_TOOL_TRIPLETAP",  # touchpad triple tap

}
_KEYBOARD_KEYS.update(_MOUSE_BUTTONS)

Just by adding these codes (given to me by ChatGPT when I asked it for ideas on modifying to code to work better with touchpad devices), I went from a strange situation where it sometimes seemed like only a 50/50 chance that Cmd+clicking (or Cmd+tapping) on links on a website would successfully open them in a new tab, to mostly success, mixed with sometimes the click (or tap) not seeming to be registered at all (which by itself is a big improvement to the click being registered as if it happened without the modifier held down and opening the link in the same page).

The ignored click doesn't seem to be me just not tapping firmly enough (which happens a lot with non-Apple cheapo touchpads) or not clicking with my finger in the right place. Seems to be more about just waiting a bit between Mod+clicking attempts. Might even be Firefox that just doesn't want to accept an additional Mod+click before the new tab has started to load its contents. Hard to say.

LibreOffice Writer seems to have no such issue with some rapid Shift+clicking to change the selection of text blocks. And the older behavior of losing the selection off and on, as a click would be interpreted as happening without the mod key, seems to be gone.

I cut the suspend timeout down to zero instead of 0.1s, and this improved behavior seems to hold up.

This seems very promising, but I just copied and pasted this from the AI suggestion and I have no real understanding of the possibility of negative side effects under other circumstances.

[RedBearAK]

Treating it separately like this also seems to work fine, adding it to the _KEYBOARD_KEYS set the same way.

# Remove all buttons so udev doesn't think keyszer is a joystick
_KEYBOARD_KEYS = ecodes.keys.keys() - ecodes.BTN

# But we want mouse buttons, so let's enumerate those and add them
# back into the set of buttons we'll watch and use
_MOUSE_BUTTONS = {
    256: ["BTN_0", "BTN_MISC"],
    257: "BTN_1",
    258: "BTN_2",
    259: "BTN_3",
    260: "BTN_4",
    261: "BTN_5",
    262: "BTN_6",
    263: "BTN_7",
    264: "BTN_8",
    265: "BTN_9",
    272: ["BTN_LEFT", "BTN_MOUSE"],
    274: "BTN_MIDDLE",
    273: "BTN_RIGHT",
}
_KEYBOARD_KEYS.update(_MOUSE_BUTTONS)

# We also want trackpad/touchpad buttons, so let's enumerate those and add them
# back into the set of buttons we'll watch and use
_TOUCHPAD_BUTTONS = {
    330: ["BTN_TOOL_FINGER", "BTN_TOUCHPAD"],                   # touchpad click
    333: "BTN_TOOL_DOUBLETAP",                                  # touchpad double tap
    334: "BTN_TOOL_TRIPLETAP",                                  # touchpad triple tap
    335: "BTN_TOOL_QUADTAP",                                    # touchpad quadruple tap
    336: "BTN_TOOL_QUINTTAP",                                   # touchpad quintuple tap
    337: "BTN_TOOL_TRIPLETAP2",                                 # second triple tap on a touchpad
    338: "BTN_TOOL_QUADTAP2",                                   # second quadruple tap on a touchpad
    339: "BTN_TOOL_QUINTTAP2",                                  # second quintuple tap on a touchpad
}
_KEYBOARD_KEYS.update(_TOUCHPAD_BUTTONS)

[RedBearAK]

I asked specifically about the main "click" line, because I thought it was odd that there was no code for SINGLETAP or something similar, but it doesn't seem to be a problem with the applications I'm testing. They don't register just a "touch" as a "tap" unless it's a very short duration that would normally be treated as a "tap". 🤷🏽‍♂️ So it doesn't seem to be a problem.

In that context, I'm not sure if any of the multi-tap event code lines are even relevant. It's probably the application that decides what to do as it sees multiple individual "taps" from the touchpad, and they never see these special multi-tap "events".

[RedBearAK]

For reference:

Is there specific information available about how BTN_TOUCH and BTN_TOUCHPAD are different?

From ChatGPT:

_Yes, BTN_TOUCH and BTNTOUCHPAD are different event codes with different meanings.

_BTNTOUCH represents a simple touch event, usually generated when a finger touches the touchpad. This event is often followed by a series of relative motion events that indicate the finger's movement on the touchpad surface.

_On the other hand, BTN_TOUCHPAD is a button press event generated when a user clicks on the touchpad surface. This event is usually preceded by a BTN_TOOLFINGER event that indicates that the touchpad is being used by a finger.

_In summary, BTN_TOUCH represents a simple touch event, while BTNTOUCHPAD represents a click event generated by pressing on the touchpad surface.

---

So this can probably just be trimmed down to the one line and still be effective. Seems to still work for me.

# We also want trackpad/touchpad buttons, so let's enumerate those and add them
# back into the set of buttons we'll watch and use
_TOUCHPAD_BUTTONS = {
    330: ["BTN_TOOL_FINGER", "BTN_TOUCHPAD"],                   # touchpad click
}
_KEYBOARD_KEYS.update(_TOUCHPAD_BUTTONS)