xcape features (double bind modifiers)

timokau commented 8 years ago

Hi, For me currently the main thing that stops me from giving wayland a try, is missing xcape support. Now https://github.com/alols/xcape/issues/67 is open for a while, and it doesn't seem like its going to come anytime soon or even is possible. Is it correct, that the windowmanager would have to implement these features? If so, have you considered implementing them?

My specific use-cas is:

Bind Caps to ctrl when I hold it, escape else
Bind Tab to alt-gr when I hold it, Tab else

ddevault commented 8 years ago

I think you should pitch this to libinput.

timokau commented 8 years ago

Thank you, I did.

timokau commented 8 years ago

They won't fix, apparently its too high-level for libinput.

ddevault commented 8 years ago

Ugh. I'll reopen this but I'm not sure how I want it to take shape in sway.

timokau commented 8 years ago

Thanks It wouldn't really be a good solution if every compositor had to implement this (although I think that especially people using tiling window managers may profit from this). But if its the only one, I'm glad that you at least consider it.

progandy commented 8 years ago

Is this even possible in sway without changing wlc? I think the whole keymapping is hidden and not accesible.

valpackett commented 7 years ago

Looks like this might be easier with wlroots.

Plus, here's an idea! Since every problem in Wayland is solved by inventing a protocol…

We need one that's kinda similar to xwayland-keyboard-grab, but:

obviously, not an exclusive grab — keypresses must be copied to the client;
the client should provide the set of keys it's interested in — the compositor can implement security policies based on that. e.g. sway can have two different permit levels, watch-modifier-keys and watch-all-keys, or just allow everyone to grab modifier keys (they're not that interesting after all) and require permit watch-keys for a keylogger;
and of course, the interface should also include a request that injects keypresses back.

Example flow:

the hypothetical wcape or waycape or whatever requests watch_keypresses(wl_seat, ['Shift_L', 'Shift_R', 'Control'])
the user presses and releases Control in a terminal window
the compositor sends an event for Control to the terminal (which ignores it), but also sends that event to wcape because it's registered for that key
wcape reads the event and requests inject_keypress('Escape')
the compositor sends an event for Escape to the terminal

Bonus idea: clients can request a "secure mode" like in macOS, so when you're entering a password, events would not be copied to wcape. (Not super necessary when only giving access to modifier keys, but would be great for, like, running a keylogger for typing statistics I guess :D)

Timidger commented 7 years ago

Seems annoying to have to define a whole protocol for this...but it is the most compositor agnostic option :\

Unless there was some other layer above libinput that's implemented that can implement this and all the compositors start using that. Though that seems mostly unnecessary and even less likely than all the compositors using the same protocol.

valpackett commented 7 years ago

Yeah, just setting xcape style key presses is simple enough to do directly in compositors, but this protocol allows for more stuff, such as TextExpander-like apps, typing statistics, etc.

valpackett commented 7 years ago

What do y'all think about this protocol?

ddevault commented 7 years ago

-1. This is not something I'm interested in doing with a protocol.

valpackett commented 7 years ago

So it won't be possible to do UI automation/testing/macros (java.awt.Robot, Sikuli, xdotool etc.), text snippet expansion, keyboard typing statistics, etc. under Sway?

I guess I'll have to do this on evdev level then. It's possible to listen directly on evdevs and create a fake keyboard (uinput on Linux, cuse on FreeBSD) to inject events. I believe that a Wayland protocol would've been a much cleaner and better solution >_<

ddevault commented 7 years ago

I don't think that any of those use-cases justify putting in what is literally a keylogger protocol extension.

valpackett commented 7 years ago

What if I want a literal keylogger to analyze my typing speed?

These features are useful. Global input access shouldn't be completely killed, it should be available to privileged applications. macOS uses accessibility APIs for this, you have to tick a checkbox in Preferences to allow an app to do this. Sway has IPC security already, which is pretty similar! permit /usr/local/bin/xcape keylogging

ddevault commented 7 years ago

Then you should patch sway.

valpackett commented 7 years ago

What do you mean by "patch"? "maintain your own fork"? :)

Actually, I'll probably do the evdev-level thing because it will work with GNOME, KDE, etc. out of the box. Would help with https://github.com/bulletmark/libinput-gestures/issues/27 as well.

ddevault commented 7 years ago

No, I mean patch. I run sway with some private patches myself. Patching software is not a big deal. Just do it until you get bored of gathering stats on your typing habits.

oblitum commented 6 years ago

I'm author of this, may be interesting on this topic:

https://gitlab.com/interception/linux/tools

Specifically the caps2esc plugin covers "Bind Caps to ctrl when I hold it, escape else" and hideaway covers "Hide mouse cursor on inactivity". I use it on Sway (works on X as well).

ddevault commented 6 years ago

That's pretty awesome!

valpackett commented 6 years ago

So, seems like the list of solutions is here: https://github.com/alols/xcape/issues/67

My solution is evscript, it runs Dyon scripts with access to evdev/uinput.

frsfnrrg commented 6 years ago

I have a patch for sway which permits key event injection (and hence xcape behavior), although there are significant limitations. It requires that one have an identifier for the keyboard (for this example, 0:0:X11_keyboard press)† , and creates a new process every use, but does not require root.

0001-Add-fake_key-command.txt

Note that faked event timestamps are 0, and please read the warning.

Then in the sway config file, add a line like

bindsym --release Control_L exec swaymsg fake_key 0:0:X11_keyboard press_and_release 9

in which 9 is the xkb keycode for Escape. swaymsg indirection avoids a crash.

† You can find this by running with full logging (sway -d), and then reading the log for a line like

2018-06-02 11:13:59 - [sway/sway/input/input-manager.c:221] adding device: '1:3:example_keyboard'

emersion commented 6 years ago

Note that you can get keyboard identifiers with swaymsg -t get_inputs.

julientechdev commented 5 years ago

I'm just going to drop my 2 cents here, I have been using this behaviour for more than a year, and it's awesome for tiling wm / vim / graphical apps with ctrl; left pinky finger has way less stress at the end of the day.

I used https://github.com/oblitum/caps2esc at first and packaged it to exherbo, then @oblitum moved it to https://gitlab.com/interception/linux/plugins/caps2esc as a plugin to a bigger interception-tools project.

Now that hideaway is built-in to sway, having such a generic input manipulation tool running just for the caps2esc behaviour seems a little overkill and being able to directly configure it from sway would be awesome.

edrex commented 5 years ago

@avrelaun if you just want to map caps to escape (rather than conditionally to ctrl or escape depending on whether chorded as caps2esc does) you can do that with a hwdb rule as in https://github.com/edrex/etc/blob/master/udev/hwdb.d/90-custom-keyboards.hwdb.

emersion commented 5 years ago

if you just want to map caps to escape (rather than conditionally to ctrl or escape depending on whether chorded as caps2esc does)

The preferred solution is to use the xkeyboard-config option.

oblitum commented 5 years ago

For chording keys I think an external tool is preferred over something builtin. It doesn't look like a task for sway. For example, caps2esc has to manipulate key-down and key-up in a specific way to make chording natural and not cause problems like missing keystrokes.

julientechdev commented 5 years ago

@edrex I need the exact behaviour with both ctrl and escape depending on the case.

jcalonso commented 5 years ago

I came here because I am looking for a way to replace this xcape config:

xcape -e 'Shift_L=Shift_L|parenleft;Shift_R=Shift_R|parenright'

Any ideas if this is possible to accomplish with Sway? or other Wayland compatible tools?

zsugabubus commented 5 years ago

Sorry for being off-topic, but if somebody has the same issue and it's the only thing that holds one back to switch, it can be super useful.

@jcalonso, not with (stable) Sway but at a lower level you can achieve it with the above mentioned Interception Tools (by @oblitum). If you could make that work and you don't want to reinvent the wheel you can use my general purpose plugin for a quick solution.

/* tap-rules.h.in */
{ .base_key   = KEY_LEFTSHIFT, /* change this key */
  .tap_key    = KEY_KPLEFTPAREN, /* what to do when pressed alone/pressed once */
  .repeat_key = KEY_KPLEFTPAREN, /* what to do when pressed for longer time (just remove it if you find it useless) */
  .hold_key   = KEY_LEFTSHIFT /* behave as this key as soon as other key pressed */  },
/* similar for right shift... */

@timokau, your config:

/* tap-rules.h.in */
{ .base_key = KEY_CAPSLOCK, .tap_key = KEY_ESC, .repeat_key = KEY_LEFTCTRL, .hold_key = KEY_LEFTCTRL },
{ .base_key = KEY_TAB, .tap_key = KEY_TAB, .hold_key = KEY_RIGHTALT/*altgr*/ },

spectras commented 5 years ago

So, what is the current state of key event copying?

I am the maintainer of https://github.com/keyleds/keyleds. It implements visual feedback on gaming keyboards that have per-key lights, triggering animations to respond to key events.

How would I go about supporting sway/wlroot/wayland, whichever is the right level?

emersion commented 5 years ago

How would I go about supporting sway/wlroot/wayland, whichever is the right level?

We've already said we're not interested in a keylogging protocol. If you want to do this, read events with libinput, like e.g. wshowkeys. This requires a privileged process, because regular processes shouldn't have a way to read all key events.

spectras commented 5 years ago

Let me get this straight:

The recommended way to have a secure environment is to force complex applications that interact with the desktop and run user-provided scripts to run as root?

I mean, that the compositor would ask the user for permission before granting access seems logical. That the feature would be completely missing…

Kommynct commented 3 years ago

Sorry to bump such an old topic but I have found that SKR

https://aur.archlinux.org/packages/skr-git/ https://github.com/DavRack/skr

Should work well for this in principle.

goetzc commented 3 years ago

@Kommynct I think a better and more mature alternative is Interception Tools which was suggested in a previous comment, using the k2k plugin.

spectras commented 3 years ago

While we're bumping that topic, any updates on how can gaming keyboard effects be implemented on Wayland?

Requirements are as follows. Keyboard management program needs:

To monitor key presses (sorry about that, but it's hard to give visual feedback to keypresses without the ability to know when keys are pressed).
To monitor either forground or focused application (lighting profiles are based on active application).
To run unprivileged.

As long as this cannot be done, there is a whole class of devices that cannot be used fully. But I did not really follow updates, so maybe I missed some news?

(I could also mention all kinds of business use cases such as screen sharing with remote control, teamviewer, automated GUI testing, robotic process automation software, that just become impossible under Wayland).

fluix-dev commented 3 years ago

While we're bumping that topic, any updates on how can gaming keyboard effects be implemented on Wayland?

Most of this is not in the scope of Wayland (the protocols) or the compositor (Sway) itself:

To monitor key presses (sorry about that, but it's hard to give visual feedback to keypresses without the ability to know when keys are pressed).

Is this for changing colors? If so, there's already a variety of Linux tools for controlling Razer and other devices and their colours such as the Openrazer drivers, alongside RazerGenie or Polychromatic GUI tools. They are perfectly capable, probably through the Openrazer drivers (though I'm not too sure), at listening to key presses and modifying the visual effects.

The main point that people point out with adding some way of monitoring keys is that it's basically a keylogging protocol, which doesn't sound great without some better security implementations first.

To monitor either forground or focused application (lighting profiles are based on active application).

The IPC already allows this, you don't really need anything else from Sway.

(I could also mention all kinds of business use cases such as screen sharing with remote control, teamviewer, automated GUI testing, robotic process automation software, that just become impossible under Wayland).

Look at something like ydotool for automated typing and clicking.

t184256 commented 3 years ago

I think it sounds like a job for remappers like kmonad, not sway.

spectras commented 3 years ago

Razer and other devices and their colours such as the Openrazer drivers, alongside RazerGenie or Polychromatic GUI tools. They are perfectly capable, probably through the Openrazer drivers (though I'm not too sure), at listening to key presses and modifying the visual effects.

Those need to run as root, and even as kernel module for some features.
Except for RazerGenie which relies on a root daemon to bypass wayland device security entirely.

That is indeed a path we could take. Is that really the security model Wayland wants to promote? Take features that could be implemented as regular user-space unprivileged code, and tell developers to make them ring0 and users to give them root permission?

Well I guess that is one way to see it, after all if you distribute the key to everyone, at least when you find your vault empty you can rejoice that is was not broken.

The main point that people point out with adding some way of monitoring keys is that it's basically a keylogging protocol

The main point is: some use cases require software to know a key was pressed.

It can already be done today, one simply has to ditch security and make everything run as root.
Or WMs can design a secure protocol, giving control to the user. “Application foo has requested permission to monitor your keypresses, [insert here whatever additional warning makes sense] — do you agree?” seems perfectly reasonable.

By refusing the latter approach, Wayland developers just force people to do the former. The result is a net decrease in overall security.

Also, having it done by the WM would make it possible to have fine control over which keypresses would be visible. Like MS Windows does, where keypresses directed to sensitive applications are not visible to hooks.

Anyway, as you see I am less active on this now. We just marked our application as incompatible with Wayland, and the editors we work with simply added "no Wayland" to their system requirements for their own software. Users are fine with that, so that works for us at the moment.

When Wayland is mature enough we can reconsider. Or just run everything as root, if it's really the Wayland's way as this conversation implies. So much for the promise of improved security.

fluix-dev commented 3 years ago

@spectras Since my comment I've read a bit more about the state of security in Wayland, so here's some stance/thoughts.

At the moment, as we've both stated, the way for direct event capture is through running something as root. I completely agree with you that this isn't ideal, but it requires at least some decision from the user.

I am against adding a protocol to globally grab key events at this moment because a protocol for proper security contexts hasn't been established yet (see here and here). Adding the key grabbing protocol before the security contexts would make unprivileged applications be able to log all keys: arguably worse than the current situation. Once a security context protocol has been merged, I think the protocol you're proposing definitely has some merit.

Note: I do not have extensive knowledge of what's discussed here, so take my comments with a grain of salt.

kennylevinsen commented 3 years ago

Indeed. Use of root is bad, but this is a net increase I'm security from X: rather than everyone and their mother being able to keylog, only root has the ability, and root is omnipotent anyway.

A Wayland protocol will be designed eventually for tasks like keybinding. We will probably still avoid full keylogging. The only reason this does not exist yet is due to other priorities.

There is nothing wrong with falling back on X until Wayland fits your use case.

spectras commented 3 years ago

Hey both of you. Thanks for taking the time and energy on this topic.

fluix-dev> I agree, having security contexts makes sense in general, and listening keypresses, being sensitive, could live behind such a mechanism. I am not familiar with wayland development. Is there a way we can help make security contexts happen?

kennylevinsen> my point is the lack of granularity (root or nothing) pushes use cases that used to be user applications to root privileges. Which incurs a decrease in security from X.

In the end, it is the whole class of "user application that needs some special gui permission" that is missing from Wayland, and "make them all root" is hardly a solution. Keypresses are just an example of such a permission. I think fluix-dev points to the right thing with security contexts, it does look like a clean approach to the more general problem.

kennylevinsen commented 3 years ago

In the end, it is the whole class of "user application that needs some special gui permission" that is missing from Wayland

We have plenty of privileged protocols already. Screen recording, layer shell, clipboard management are all protocols that are not intended for general use and, at some point, will be subject to restriction.

A dedicated key binding protocol would be in the same category and require approval, but a omnipotent key logging protocol is unwanted as abuse is unavoidable (just one popular proprietary application misusing it for key bindings is enough) and users would end up unable or unwilling to restrict it, leading to a security situation worse than what we have now: unprivileged keylogging free-for-all.

A key binding protocol is pretty easy to write, but as the primary usecases so far have all been proprietary applications such as discord and zoom that we cannot work on an implementation for (working implementations are a requirement to contribute protocols), no work has To my knowledge been made on this subject. It should be quite a simple protocol though.

alex-courtis commented 2 years ago

dual-function-keys should fill your requirements.

It's a plugin for interception tools, which captures and operates directly on udev/evdev devices.

t0ster30 commented 2 years ago

Try evdoublebind — it works like xcape, except it uses evdev as a backend (so it can be used both in Sway and i3). Some xkb parts need to be set up though.

Kommynct commented 2 years ago

I now prefer keyd for everything related to this, there seems to be many options now for this, this should probably be closed

swaywm / sway

xcape features (double bind modifiers) #719

Sorry for being off-topic, but if somebody has the same issue and it's the only thing that holds one back to switch, it can be super useful.