iberianpig / fusuma-plugin-sendkey

Fusuma plugin that sending virtual keyboard events
MIT License
72 stars 2 forks source link

plugin seems to not be working #26

Closed Sorunome closed 2 months ago

Sorunome commented 2 years ago

Using awesome WM, the sendkey plugin seems to be doing nothing.

Here is the ~/.config/fusuma/config.yml:

plugin:
  executors:
    wmctrl_executor:
      wrap-navigation: true

swipe:
  3:
    left:
      workspace: "next"
    right:
      workspace: "prev"
    up:
      sendkey: "LEFTCTRL+T"

threshold: 
  swipe: 0.4 
  pinch: 0.2 

interval: 
  swipe: 1.0 
  pinch: 0.2

The logging does show e.g. I, [2022-05-30T14:59:57.270877 #30258] INFO -- : {:sendkey=>"LEFTCTRL+T"}, however e.g. firefox does not open a new tab.

The output of fusuma -l is:

I, [2022-05-30T15:00:26.293697 #30289]  INFO -- : reload config: /home/sorunome/.config/fusuma/config.yml
I, [2022-05-30T15:00:26.319017 #30289]  INFO -- : ---------------------------------------------
I, [2022-05-30T15:00:26.319152 #30289]  INFO -- : Fusuma: 2.3.0
I, [2022-05-30T15:00:26.319916 #30289]  INFO -- : libinput: 1.20.1
I, [2022-05-30T15:00:26.319932 #30289]  INFO -- : ruby 3.0.4p208
I, [2022-05-30T15:00:26.320132 #30289]  INFO -- : OS: Linux 5.17.0-rc7-asahi-next-20220310-5-2-ARCH #2 SMP PREEMPT Tue, 22 Mar 2022 14:28:10 +0000
I, [2022-05-30T15:00:26.320336 #30289]  INFO -- : Distribution: Arch Linux \r (\l)
I, [2022-05-30T15:00:26.321496 #30289]  INFO -- : Desktop session: awesome x11
I, [2022-05-30T15:00:26.321503 #30289]  INFO -- : ---------------------------------------------
I, [2022-05-30T15:00:26.321508 #30289]  INFO -- : Enabled Plugins: 
I, [2022-05-30T15:00:26.321545 #30289]  INFO -- :   Fusuma::Plugin::Buffers::GestureBuffer
I, [2022-05-30T15:00:26.321549 #30289]  INFO -- :   Fusuma::Plugin::Buffers::TimerBuffer
I, [2022-05-30T15:00:26.321554 #30289]  INFO -- :   Fusuma::Plugin::Detectors::HoldDetector
I, [2022-05-30T15:00:26.321557 #30289]  INFO -- :   Fusuma::Plugin::Detectors::PinchDetector
I, [2022-05-30T15:00:26.321561 #30289]  INFO -- :   Fusuma::Plugin::Detectors::RotateDetector
I, [2022-05-30T15:00:26.321565 #30289]  INFO -- :   Fusuma::Plugin::Detectors::SwipeDetector
I, [2022-05-30T15:00:26.321569 #30289]  INFO -- :   Fusuma::Plugin::Events::Records::ContextRecord
I, [2022-05-30T15:00:26.321572 #30289]  INFO -- :   Fusuma::Plugin::Events::Records::GestureRecord
I, [2022-05-30T15:00:26.321576 #30289]  INFO -- :   Fusuma::Plugin::Events::Records::IndexRecord
I, [2022-05-30T15:00:26.321580 #30289]  INFO -- :   Fusuma::Plugin::Events::Records::TextRecord
I, [2022-05-30T15:00:26.321583 #30289]  INFO -- :   Fusuma::Plugin::Executors::CommandExecutor
I, [2022-05-30T15:00:26.321587 #30289]  INFO -- :   Fusuma::Plugin::Executors::SendkeyExecutor
I, [2022-05-30T15:00:26.321591 #30289]  INFO -- :   Fusuma::Plugin::Executors::WmctrlExecutor
I, [2022-05-30T15:00:26.321595 #30289]  INFO -- :   Fusuma::Plugin::Filters::LibinputDeviceFilter
I, [2022-05-30T15:00:26.321598 #30289]  INFO -- :   Fusuma::Plugin::Inputs::LibinputCommandInput
I, [2022-05-30T15:00:26.321602 #30289]  INFO -- :   Fusuma::Plugin::Inputs::TimerInput
I, [2022-05-30T15:00:26.321606 #30289]  INFO -- :   Fusuma::Plugin::Parsers::LibinputGestureParser
I, [2022-05-30T15:00:26.321609 #30289]  INFO -- : ---------------------------------------------
Apple Internal Keyboard / Trackpad

Any help or advice would be greatly appreciated

iberianpig commented 2 years ago

Could you try $ fusuma-sendkey LEFTCTRL+T on terminal?

Sorunome commented 2 years ago

i tried that as sleep 3 && fusuma-sendkey LEFTCTRL+T so that i have time to switch to firefox, but also no keys were triggered

iberianpig commented 2 years ago

Do you have any errors in terminal?

Please update fusuma and all fusuma-plugins. (I noticed you are using fusuma v2.3.0)

Sorunome commented 2 years ago

Seems like the AUR for fusuma was slightly out-of-date then. Anyhow, I updated fusuma and its plugins:

fusuma (2.4.1, 2.3.0)
fusuma-plugin-sendkey (0.6.4)
fusuma-plugin-wmctrl (1.1.0)

sadly the same issue still persists, and there is no error or output at all in the terminal.

Possibly worth noting is that I am on an M1 macbook with asahi linux, however since xorg seems to be working fine and the other plugin works, sendkey should maybe also work? I'm not too sure how this stuff works internally ^^"

iberianpig commented 2 years ago

I believe fusuma-plugin-sendkey will work in your environment.

Fusuma sendkey sends events using evdev protocol. Evdev is generic event protocol for Linux input devices. So your real Apple keyboard should also send key events using evdev.

You can see the raw events of evdev in $ cat /dev/input/eventXX. (device file, XX is auto-assigned number) Also, since libinput uses evdev as its source, it is useful to know keys code and event device's number sent by libinput debug-events --show-keycodes or libinput list-devices

$ libinput list-devices ``` Device: Video Bus Kernel: /dev/input/event11 Group: 1 Seat: seat0, default Capabilities: keyboard Tap-to-click: n/a Tap-and-drag: n/a Tap drag lock: n/a Left-handed: n/a Nat.scrolling: n/a Middle emulation: n/a Calibration: n/a Scroll methods: none Click methods: none Disable-w-typing: n/a Accel profiles: n/a Rotation: n/a Device: Power Button Kernel: /dev/input/event1 Group: 2 Seat: seat0, default Capabilities: keyboard Tap-to-click: n/a Tap-and-drag: n/a Tap drag lock: n/a Left-handed: n/a Nat.scrolling: n/a Middle emulation: n/a Calibration: n/a Scroll methods: none Click methods: none Disable-w-typing: n/a Accel profiles: n/a Rotation: n/a Device: Lid Switch Kernel: /dev/input/event0 Group: 3 Seat: seat0, default Capabilities: switch Tap-to-click: n/a Tap-and-drag: n/a Tap drag lock: n/a Left-handed: n/a Nat.scrolling: n/a Middle emulation: n/a Calibration: n/a Scroll methods: none Click methods: none Disable-w-typing: n/a Accel profiles: n/a Rotation: n/a Device: Sleep Button Kernel: /dev/input/event2 Group: 4 Seat: seat0, default Capabilities: keyboard Tap-to-click: n/a Tap-and-drag: n/a Tap drag lock: n/a Left-handed: n/a Nat.scrolling: n/a Middle emulation: n/a Calibration: n/a Scroll methods: none Click methods: none Disable-w-typing: n/a Accel profiles: n/a Rotation: n/a Device: Integrated_Webcam_HD: Integrate Kernel: /dev/input/event13 Group: 5 Seat: seat0, default Capabilities: keyboard Tap-to-click: n/a Tap-and-drag: n/a Tap drag lock: n/a Left-handed: n/a Nat.scrolling: n/a Middle emulation: n/a Calibration: n/a Scroll methods: none Click methods: none Disable-w-typing: n/a Accel profiles: n/a Rotation: n/a Device: Integrated_Webcam_HD: Integrate Kernel: /dev/input/event14 Group: 5 Seat: seat0, default Capabilities: keyboard Tap-to-click: n/a Tap-and-drag: n/a Tap drag lock: n/a Left-handed: n/a Nat.scrolling: n/a Middle emulation: n/a Calibration: n/a Scroll methods: none Click methods: none Disable-w-typing: n/a Accel profiles: n/a Rotation: n/a Device: DLL0945:00 06CB:CDE6 Mouse Kernel: /dev/input/event5 Group: 6 Seat: seat0, default Capabilities: pointer Tap-to-click: n/a Tap-and-drag: n/a Tap drag lock: n/a Left-handed: disabled Nat.scrolling: disabled Middle emulation: n/a Calibration: n/a Scroll methods: *button Click methods: none Disable-w-typing: n/a Accel profiles: flat *adaptive Rotation: n/a Device: DLL0945:00 06CB:CDE6 Touchpad Kernel: /dev/input/event10 Group: 6 Seat: seat0, default Size: 108x60mm Capabilities: pointer gesture Tap-to-click: disabled Tap-and-drag: enabled Tap drag lock: disabled Left-handed: disabled Nat.scrolling: disabled Middle emulation: disabled Calibration: n/a Scroll methods: *two-finger edge Click methods: *button-areas clickfinger Disable-w-typing: enabled Accel profiles: flat *adaptive Rotation: n/a Device: Intel HID events Kernel: /dev/input/event8 Group: 7 Seat: seat0, default Capabilities: keyboard Tap-to-click: n/a Tap-and-drag: n/a Tap drag lock: n/a Left-handed: n/a Nat.scrolling: n/a Middle emulation: n/a Calibration: n/a Scroll methods: none Click methods: none Disable-w-typing: n/a Accel profiles: n/a Rotation: n/a Device: Intel HID 5 button array Kernel: /dev/input/event9 Group: 8 Seat: seat0, default Capabilities: keyboard Tap-to-click: n/a Tap-and-drag: n/a Tap drag lock: n/a Left-handed: n/a Nat.scrolling: n/a Middle emulation: n/a Calibration: n/a Scroll methods: none Click methods: none Disable-w-typing: n/a Accel profiles: n/a Rotation: n/a Device: Dell WMI hotkeys Kernel: /dev/input/event4 Group: 9 Seat: seat0, default Capabilities: keyboard Tap-to-click: n/a Tap-and-drag: n/a Tap drag lock: n/a Left-handed: n/a Nat.scrolling: n/a Middle emulation: n/a Calibration: n/a Scroll methods: none Click methods: none Disable-w-typing: n/a Accel profiles: n/a Rotation: n/a Device: AT Translated Set 2 keyboard Kernel: /dev/input/event3 Group: 10 Seat: seat0, default Capabilities: keyboard Tap-to-click: n/a Tap-and-drag: n/a Tap drag lock: n/a Left-handed: n/a Nat.scrolling: n/a Middle emulation: n/a Calibration: n/a Scroll methods: none Click methods: none Disable-w-typing: n/a Accel profiles: n/a Rotation: n/a Device: EP0430M09 Kernel: /dev/input/event6 Group: 11 Seat: seat0, default Capabilities: touch Tap-to-click: n/a Tap-and-drag: n/a Tap drag lock: n/a Left-handed: n/a Nat.scrolling: n/a Middle emulation: n/a Calibration: identity matrix Scroll methods: none Click methods: none Disable-w-typing: n/a Accel profiles: n/a Rotation: n/a Device: xremap pid=2163858 Kernel: /dev/input/event7 Group: 12 Seat: seat0, default Capabilities: keyboard pointer Tap-to-click: n/a Tap-and-drag: n/a Tap drag lock: n/a Left-handed: disabled Nat.scrolling: disabled Middle emulation: disabled Calibration: n/a Scroll methods: button Click methods: none Disable-w-typing: n/a Accel profiles: flat *adaptive Rotation: n/a ```
$ libinput debug-events --show-keycodes ``` -event11 DEVICE_ADDED Video Bus seat0 default group1 cap:k -event1 DEVICE_ADDED Power Button seat0 default group2 cap:k -event0 DEVICE_ADDED Lid Switch seat0 default group3 cap:S -event2 DEVICE_ADDED Sleep Button seat0 default group4 cap:k -event13 DEVICE_ADDED Integrated_Webcam_HD: Integrate seat0 default group5 cap:k -event14 DEVICE_ADDED Integrated_Webcam_HD: Integrate seat0 default group5 cap:k -event5 DEVICE_ADDED DLL0945:00 06CB:CDE6 Mouse seat0 default group6 cap:p left scroll-nat scroll-button -event10 DEVICE_ADDED DLL0945:00 06CB:CDE6 Touchpad seat0 default group6 cap:pg size 108x60mm tap(dl off) left scroll-nat scroll-2fg-edge click-buttonareas-clickfinger dwt-on -event8 DEVICE_ADDED Intel HID events seat0 default group7 cap:k -event9 DEVICE_ADDED Intel HID 5 button array seat0 default group8 cap:k -event4 DEVICE_ADDED Dell WMI hotkeys seat0 default group9 cap:k -event3 DEVICE_ADDED AT Translated Set 2 keyboard seat0 default group10 cap:k -event6 DEVICE_ADDED EP0430M09 seat0 default group11 cap:t ntouches 5 calib -event3 KEYBOARD_KEY +0.000s KEY_A (30) pressed a event3 KEYBOARD_KEY +0.107s KEY_A (30) released event3 KEYBOARD_KEY +0.573s KEY_RIGHTSHIFT (54) pressed event3 KEYBOARD_KEY +0.740s KEY_A (30) pressed A event3 KEYBOARD_KEY +0.863s KEY_A (30) released event3 KEYBOARD_KEY +1.309s KEY_RIGHTSHIFT (54) released event3 KEYBOARD_KEY +4.812s KEY_A (30) pressed a event3 KEYBOARD_KEY +4.922s KEY_A (30) released event3 KEYBOARD_KEY +5.899s KEY_LEFTCTRL (29) pressed event3 KEYBOARD_KEY +6.438s KEY_C (46) pressed ```

event3 is my keyboard device number, /dev/inputs/event3 is device file. fusuma-sendkey writes key events to event device (/dev/inputs/event3), it behaves like real keyboard events.

Could you paste output of libinput list-devices and libinput debug-events --show-keycodes?

fneirotti commented 2 years ago

I had a similar problem, and after expending the better part of yesterday investigating the cause, I found something that may be useful for @Sorunome or anyone else that comes across a similar issue.

libinput list-devices ``` Device: Video Bus Kernel: /dev/input/event3 Group: 1 Seat: seat0, default Capabilities: keyboard Tap-to-click: n/a Tap-and-drag: n/a Tap drag lock: n/a Left-handed: n/a Nat.scrolling: n/a Middle emulation: n/a Calibration: n/a Scroll methods: none Click methods: none Disable-w-typing: n/a Disable-w-trackpointing: n/a Accel profiles: n/a Rotation: n/a Device: Lid Switch Kernel: /dev/input/event0 Group: 2 Seat: seat0, default Capabilities: switch Tap-to-click: n/a Tap-and-drag: n/a Tap drag lock: n/a Left-handed: n/a Nat.scrolling: n/a Middle emulation: n/a Calibration: n/a Scroll methods: none Click methods: none Disable-w-typing: n/a Disable-w-trackpointing: n/a Accel profiles: n/a Rotation: n/a Device: Power Button Kernel: /dev/input/event1 Group: 3 Seat: seat0, default Capabilities: keyboard Tap-to-click: n/a Tap-and-drag: n/a Tap drag lock: n/a Left-handed: n/a Nat.scrolling: n/a Middle emulation: n/a Calibration: n/a Scroll methods: none Click methods: none Disable-w-typing: n/a Disable-w-trackpointing: n/a Accel profiles: n/a Rotation: n/a Device: Sleep Button Kernel: /dev/input/event2 Group: 4 Seat: seat0, default Capabilities: keyboard Tap-to-click: n/a Tap-and-drag: n/a Tap drag lock: n/a Left-handed: n/a Nat.scrolling: n/a Middle emulation: n/a Calibration: n/a Scroll methods: none Click methods: none Disable-w-typing: n/a Disable-w-trackpointing: n/a Accel profiles: n/a Rotation: n/a Device: Asus Keyboard Kernel: /dev/input/event4 Group: 5 Seat: seat0, default Capabilities: keyboard pointer Tap-to-click: n/a Tap-and-drag: n/a Tap drag lock: n/a Left-handed: n/a Nat.scrolling: disabled Middle emulation: n/a Calibration: n/a Scroll methods: none Click methods: none Disable-w-typing: n/a Disable-w-trackpointing: n/a Accel profiles: n/a Rotation: n/a Device: Asus Keyboard Kernel: /dev/input/event5 Group: 5 Seat: seat0, default Capabilities: keyboard Tap-to-click: n/a Tap-and-drag: n/a Tap drag lock: n/a Left-handed: n/a Nat.scrolling: n/a Middle emulation: n/a Calibration: n/a Scroll methods: none Click methods: none Disable-w-typing: n/a Disable-w-trackpointing: n/a Accel profiles: n/a Rotation: n/a Device: Asus Keyboard Kernel: /dev/input/event6 Group: 5 Seat: seat0, default Capabilities: pointer Tap-to-click: n/a Tap-and-drag: n/a Tap drag lock: n/a Left-handed: disabled Nat.scrolling: disabled Middle emulation: disabled Calibration: n/a Scroll methods: button Click methods: none Disable-w-typing: n/a Disable-w-trackpointing: n/a Accel profiles: flat *adaptive Rotation: n/a Device: USB2.0 HD UVC WebCam: USB2.0 HD Kernel: /dev/input/event7 Group: 6 Seat: seat0, default Capabilities: keyboard Tap-to-click: n/a Tap-and-drag: n/a Tap drag lock: n/a Left-handed: n/a Nat.scrolling: n/a Middle emulation: n/a Calibration: n/a Scroll methods: none Click methods: none Disable-w-typing: n/a Disable-w-trackpointing: n/a Accel profiles: n/a Rotation: n/a Device: USB2.0 HD UVC WebCam: USB2.0 IR Kernel: /dev/input/event8 Group: 6 Seat: seat0, default Capabilities: keyboard Tap-to-click: n/a Tap-and-drag: n/a Tap drag lock: n/a Left-handed: n/a Nat.scrolling: n/a Middle emulation: n/a Calibration: n/a Scroll methods: none Click methods: none Disable-w-typing: n/a Disable-w-trackpointing: n/a Accel profiles: n/a Rotation: n/a Device: ASUE120A:00 04F3:319B Mouse Kernel: /dev/input/event12 Group: 7 Seat: seat0, default Capabilities: pointer Tap-to-click: n/a Tap-and-drag: n/a Tap drag lock: n/a Left-handed: disabled Nat.scrolling: disabled Middle emulation: n/a Calibration: n/a Scroll methods: *button Click methods: none Disable-w-typing: n/a Disable-w-trackpointing: n/a Accel profiles: flat *adaptive Rotation: n/a Device: ASUE120A:00 04F3:319B Touchpad Kernel: /dev/input/event13 Group: 7 Seat: seat0, default Size: 126x74mm Capabilities: pointer gesture Tap-to-click: disabled Tap-and-drag: enabled Tap drag lock: disabled Left-handed: disabled Nat.scrolling: disabled Middle emulation: disabled Calibration: n/a Scroll methods: *two-finger edge Click methods: *button-areas clickfinger Disable-w-typing: enabled Disable-w-trackpointing: enabled Accel profiles: flat *adaptive Rotation: n/a Device: Asus WMI hotkeys Kernel: /dev/input/event10 Group: 8 Seat: seat0, default Capabilities: keyboard Tap-to-click: n/a Tap-and-drag: n/a Tap drag lock: n/a Left-handed: n/a Nat.scrolling: n/a Middle emulation: n/a Calibration: n/a Scroll methods: none Click methods: none Disable-w-typing: n/a Disable-w-trackpointing: n/a Accel profiles: n/a Rotation: n/a ```

A little bit of backstory first (go to the end for the solution):

I had been using Fusuma for a while without problems, but when I changed my laptop it stopped working altogether. The main Fusuma process and fusuma-sendkey both started without any errors, but the keypresses were not being sent.

So after googling for an answer and not finding any, I decided to download this repo and debug a little bit what was happening. With the repo on one side and libinput list-devices on the other, I found that fusuma-sendkey sends the keycodes to the first device that matches a keyboard regex and on my system there were 3(!) of them.

This finding led me to manually set other inputs (such as input/event5) and run fusuma-sendkey with it. And it worked!

Afterwards, I went into a rabbit hole of documentation and blogposts talking about inputs and udev rules that led me nowhere. I tried to rename the input (not possible) and later to remove it completely (or at least not recognized by libinput).

But I discovered something: certain keys were only being handled by input/event4 and others by input/event5 and some of them by both!. For example, KEY_KBDILLUMUP worked only on event4, any alphabet letter only on event5 and KEY_F21 on both.

With this newfound knowledge, I remembered other tools that I had tried before and worked, such as ydotool. Ydotool is set up a little differently in that it requires a constantly running daemon to function. Looking into its code I found that it was creating a new virtual input device when starting the daemon.

So my solution was to create a dummy virtual keyboard and configure it in fusuma.yml (sendkey_executor:device_name) to point to that device.

For that, I made a simple console tool that can be run as a daemon to create a virtual keyboard.

BirsanStefan commented 1 year ago

@fneirotti that is a great answer, it helped me find a solution. Thanks a lot!

Running 'libinput list-devices' displays all devices, then I tried a different one that seemd like a keyboard.


plugin:
  executors:
    sendkey_executor:
      device_name: 'AT Translated Set 2 keyboard'
swipe
  3: ........
gwydion67 commented 8 months ago

that really helped @fneirotti. thenks i think this should be included in readme. laptop devices often have multiple "keyboard" type devices and the plugin seems to pick whichever is the first ( as in output of libinput list-devices), therefore not being able to to do anything

iberianpig commented 8 months ago

fusuma-plugin-remap is still in development and experimental, but it can be used alongside fusuma-plugin-sendkey. When you install fusuma-plugin-remap, fusuma creates a dedicated virtual keyboard and fusuma-plugin-sendkey use it automatically. This eliminates the need to manually specify a keyboard in yaml, simplifying the setup process.

iberianpig commented 2 months ago

By installing fusuma-plugin-remap, a virtual device will be automatically created.

$ libinput list-devices
...
Device:           fusuma_virtual_keyboard
Kernel:           /dev/input/event14
Group:            12
Seat:             seat0, default
Capabilities:     keyboard pointer
Tap-to-click:     n/a
Tap-and-drag:     n/a
Tap drag lock:    n/a
Left-handed:      disabled
Nat.scrolling:    disabled
Middle emulation: disabled
Calibration:      n/a
Scroll methods:   button
Click methods:    none
Disable-w-typing: n/a
Disable-w-trackpointing: n/a
Accel profiles:   flat *adaptive custom
Rotation:         0.0

Fusuma will use fusuma_virtual_keyboard if found it. This will be the default for future versions of Fusuma. If you still encounter issues after setting up fusuma-plugin-remap, please let us know again.