Run evsieve with persist=reopen even if the input device doesn't exist?

I'd like to run the following code at startup even if the input device doesn't yet exist, rather than waiting for the input device:

sudo evsieve --input /dev/input/by-id/original-keyboard grab persist=reopen --output create-link=/dev/input/by-id/virtual-keyboard

With persist=reopen, you can disconnect it once the process is ran, but why not allow it to be initially disconnected also before starting the process?

Thank you for your suggestion.

This is already a planned feature for a future version of evsieve, where "future" likely means version 1.4 or 1.5, though the clause to get that behaviour will then be called persist=full or persist or something. The reopen part exists to explicitly signal that it requires the device to be available when evsieve starts.

To explain why this feature is not implemented yet: this feature is more complicated than it initially looks because event devices have a thing called "capabilities": at the time of creation, event devices needs to announce which kinds of events they may possibly generate. After creation, these capabilities cannot be changed without destroying the device and creating another one.

For example, a keyboard may announce that it can generate EV_KEY events with codes for every button it has, while a mouse may announce that it can generate EV_REL events for the X and Y axes plus some EV_KEY events for however many buttons it has. You can check which capabilities your device has using the evtest program (probably available on your distro's repositories.)

When evsieve starts, it automatically computes which events its virtual output devices can possibly generate based on the capabilities of the input devices and all transformations applied to the events. For most intents and purposes, the user never has to worry about capabilities because "it just works".

This story becomes different when the input devices are not actually available when evsieve starts, because without knowing what events the input devices generate, there is no way to compute what kind of events the virtual devices can possibly generate. There are some ways to deal with this, each with severe drawbacks:

Approach 1: the virtual output device can be destroyed and recreated when it turns out that it can generate more event types than previously thought. Drawbacks: some applications like Qemu will not reopen virtual devices you pass to them; if evsieve destroys and recreates a device, it will look like Qemu closed a device for no reason.

Approach 2: evsieve can decide not to create an output device until (all?) input devices that can possibly generate events for it are available. Drawbacks: mostly the same as the previous approach. Also has questionable behaviour when multiple input devices can generate events for the same virtual output device.

Approach 3: the virtual device can announce that it can produce every event code imaginable. Drawbacks: this may confuse other applications which will think that your computer now has a second keyboard and a second mouse and a joystick and a touchscreen and a pentablet and a lot more. Also, this is infeasible for EV_ABS-type events because for those events you need to not only provide which event codes it can produce, but the size of the axes as well.

Approach 4: if evsieve sees a device for the first time, it can create a file in a configuration directory (/var/lib/evsieve I suppose?) which remembers what the capabilities of that device were. If a given device is not available when the program starts a later time, it reads that configuration file and will assume that the missing device will have those capabilities when it shows up later. Drawbacks: requires the devices to be available the first time the script runs, requires a configuration directory and stores files on the users' computer. Also mysteriously breaks if the capabilities of the input device turn out to be different from what they were the last time.

So far I think that the last approach has the least severe drawbacks, or at least is the only approach that is viable for one of evsieve's primary use cases.

I may or may not decide to fall back on the first approach if the cached capabilities turn out to be incorrect. On one hand, if a mouse shows up where a keyboard was expected, recreating the virtual devices is most sensible thing to do. On the other hand, if a kernel update decides to add a new obscure capability to your keyboard that you will probably not use, recreating the virtual devices could cause a user's virtual machine to unexpectedly and unnecessarily lose access to its keyboard one time.

I could ask the user which approach they want (i.e. asking them to specify persist=cache or persist=recreate) since both strategies are good for different use cases, but I would rather not bother the user too much with low-level gritty details. Anyway, the design process around this feature is still not quite done yet.

I appreciate the explanation very much!

Approach 4: if evsieve sees a device for the first time, it can create a file in a configuration directory (/var/lib/evsieve I suppose?) which remembers what the capabilities of that device were. If a given device is not available when the program starts a later time, it reads that configuration file and will assume that the missing device will have those capabilities when it shows up later. Drawbacks: requires the devices to be available the first time the script runs, requires a configuration directory and stores files on the users' computer. Also mysteriously breaks if the capabilities of the input device turn out to be different from what they were the last time.

This is definitely the most appealing approach to me. I can't see how the "first time" requirement would ever be a big deal.

I may or may not decide to fall back on the first approach if the cached capabilities turn out to be incorrect. On one hand, if a mouse shows up where a keyboard was expected, recreating the virtual devices is most sensible thing to do. On the other hand, if a kernel update decides to add a new obscure capability to your keyboard that you will probably not use, recreating the virtual devices could cause a user's virtual machine to unexpectedly and unnecessarily lose access to its keyboard one time.

From an end user's standpoint, I'd probably want a notification with a warning message that gives an option to either recreate it or let it be. The latter being the default if the user ignores the notification. Maybe provide a cli command to recreate it.

I could ask the user which approach they want (i.e. asking them to specify persist=cache or persist=recreate) since both strategies are good for different use cases, but I would rather not bother the user too much with low-level gritty details. Anyway, the design process around this feature is still not quite done yet.

Yeah, I think the best way to go about it is to try to develop a solution that "fits all" so that you only need a simple persist flag. Keep it as simple as possible for maintenance sake. Only when you know that it's impossible to have a one-size-fits-all solution should you then offer options.

From an end user's standpoint, I'd probably want a notification with a warning message that gives an option to either recreate it or let it be. The latter being the default if the user ignores the notification. Maybe provide a cli command to recreate it.

After some thought, I don't think this is a viable approach because:

Evsieve is usually ran as a background service or as part of a script; while evsieve is running, the user is probably not staring at the terminal to see if it asks the user something;
Even if I could work around that issue by using some kind of D-Bus notifications API and the user is looking at his Linux desktop when the notification shows up, the kind of user who is looking at his Linux desktop tends to be the kind of user for who recreating an input device is not a problem (it is mostly a problem if the user is using a VM);
Also, there is no reasonable timeframe which is long enough for the user could figure out the right answer to this obscure notification, yet short enough to not stall the input system for an undue amount of time in case device recreation is necessary.

My new idea is: whether an output device can be destroyed and recreated to satisfy new capability requirements, should be a property of the output device and not of the input device's mode of persistence. E.g. I could add a hypothetical pin flag to an output device like:

evsieve --input /dev/input/by-id/keyboard persist \
        --output pin

Which would instruct evsieve that this "pinned" output device should under no circumstances be destroyed and recreated. In case an output device gets destroyed and recreated, evsieve should log (print) a message that this happened and informs the user that they can add the pin to their output device to prevent this from ever happening again.

Keep in mind that needing to destroy and recreate output devices is a "once in a blue moon" kind of situation. The four things I can think of that can cause it are (1) bugs in evsieve, (2) kernel/driver updates, (3) referring to devices in an unreliable way, e.g. like --input /dev/input/event4, and (4) the user actively messing with device identification, such as:

ln -s /dev/input/by-id/keyboard ~/.config/keyboard
systemd-run evsieve --input ~/.config/keyboard grab persist=reopen --output
unlink ~/.config/keyboard
ln -s /dev/input/by-id/mouse ~/.config/keyboard

Now the big design questions left are:

1. Should output devices be pinned by default?

In some practical sense, the heuristic "anonymous output devices are not pinned by default, output devices which have a path= clause specified are pinned by default" would make the right decision in most of the cases.

However, it is an inelegant policy which can have some unexpected side effects, e.g. the user might not expect that adding/removing a path= clause could potentially break their script. Also, it really panders to the fact that Qemu is not capable of reopening closed devices. If Qemu were to ever fix that, this policy would become suboptimal.

(Now I think about it, maybe I should be sending patches to Qemu to make it capable of reopening closed devices instead of working on third party tools to mitigate Qemu's shortcomings.)

2. If an output device cannot be recreated, should the input device still be reopened/grabbed?

This is mainly relevant if some output devices are implicitly pinned. If an input device is reopened, grabbed, but unable to write its events to an output device due to the output device having insufficient capabilities, the user's input system would be locked up. However, not reopening an input device would have pretty much the same/worse consequences as not pinning an output device in the first place.

Now that I wrote this, I realised that there is actually no good answer to question 2, which means that implicitly pinned output devices are simply a bad idea, which immediately answers question 1. It's amazing how you sometimes realise the answer to your questions the moment you try to talk about them.

I'd like to run the following code at startup even if the input device doesn't yet exist, rather than waiting for the input device:

sudo evsieve --input /dev/input/by-id/original-keyboard grab persist=reopen --output create-link=/dev/input/by-id/virtual-keyboard

This would indeed be a(nother) nice feature to have.

In the meantime tho, perhaps a solution is to use evsieve in combination with persistent-evdev. I discovered this tool recently and have used it to e.g. solve the problem of switching between wired and wireless mode on my mouse without losing input to a Qemu guest. It seems to work quite well in combination with evsieve.

If you add the virtual devices that you have created with evsieve to persistent-evdevs configuration file, then the virtual device created by the latter will always be available even when the original device doesn't exist. Pass it to your virtual machine and the device will become available whenever it's ready.

I am finding myself in a very similar situation to the one described in this issue.

I have a 2-in-1 laptop/tablet where the keyboard/touchpad combo is detachable and requiring evsieve to work correctly.

Being able to start evsieve even when the keyboard is detached would be a win. Otherwise the only sensible thing to do seems to be launching evsieve from udev and having it exit automatically when the keyboard goes away.

To me the approach 4 seems sensible. I would implement it using a persist=from:file where file has the capabilities required by the input device. IMHO evsieve should not start (and log an error) if the file is absent. An additional script could then be made available to create the capability file externally from evsieve itself (e.g. evprobe device capfile).

Hopefully this could keep complexity to a minimum and would enable capability files to be prepared before-hand (e.g. as part of a support package for a specific machine or usage case).

Could this approach be at least an initial stopgap?

In the main branch (as of e6f2f4a4e3c034599bfca4411e80843599743238) I've implemented persist=full mode using the original plan of Approach 4, where evsieve automatically creates the cache files in its own directory. I'm not really that happy with the current solution, but taking a perfectionist attitude has delayed this feature for a just unreasonably long time.

Currently evsieve will print a warning and then start anyway even if neither the device nor the cached capabilities are present, with the caveat that some output device will almost certainly get destroyed and recreated when the input device actually shows up. Still not sure if that is the right approach or not. Reasons for starting anyway are:

If evsieve starts anyway, evsieve will be able to create the capabilites file as soon as the input device actually shows up;
For some use cases it doesn't actually matter much if output device gets destroyed and recreated.

Asking the user to manually create the capabilities file using something like an evprobe utility would be another potential solution as well. The advantage is that evsieve will no longer automatically pollute the filesystem with its cache files.

The disadvantage is that it complicates usage for the end user; now the user would need to do a multi-step process of "for each device create a capabilities file, then specify the path according to ..." instead of just "add the persist flag". I can already imagine somebody writing their own wrapper script around evsieve to automate the creation of the cache files.

Will you release a new version with the persist=full mode enabled? I tried out the current build and grabbing the devices, even if present when starting, results in failed to grab input device: received libevdev status code -16

Replying to myself: The failure to grab the input device was because "persistent-evdev" which I used previously to emulate the input devices not yet present when starting "evsieve" was still running. After disabling "persistent-evdev", "evsieve" with persist=full seems to be a workable alternative for this situation. Nevertheless; how about a new release with this option (and e.g. the --config option, which as of now still has to be compiled with a separate flag)?

KarsMulder / evsieve

Run evsieve with persist=reopen even if the input device doesn't exist? #2