RedBearAK / toshy

Keymapper config to make Linux work like a 'Tosh!
https://toshy.app
GNU General Public License v3.0
226 stars 15 forks source link

Pressing/holding the "fn" key should try bring up an emoji picker. #280

Closed Alberto-vgdd closed 1 month ago

Alberto-vgdd commented 1 month ago

Is your feature request related to a problem? Please describe. This feature request doesn't really correspond to a problem.

Describe the solution you'd like In MacOS, the "fn" key (bottom-left most key) is by default mapped to bring up a an emoji-selector pop up. It would be amazing if toshy could achieve something similar out of the box. It would also be cool, if the feature could be toggled on/off from the Preferences UI like some of the other settings.

Describe alternatives you've considered If something like that is unreasonable to include for the scope of this project, it would be nice if at least Toshy would map the key to the equivalent of the DE/distro that was running. In the case of GNOME, the shortcut is Ctrl + ; and brings up the image below.

Screenshot from 2024-05-16 21-19-38

Additional context I can imagine a popup like that is not standardized across DEs, and that is probably the reason why this feature hasn't been planned. But still, I would like to hear if this was considered at some point.

RedBearAK commented 1 month ago

@Alberto-vgdd

I didn't know about the GNOME shortcut. I notice that works for me (on the virtualized Cmd+Semicolon position), but only in a GTK app window like GNOME Text Editor. It's not working here in a Firefox window text area, or in VSCode (Electron/Chromium). I had been using a plugin for Ulauncher that lets me look for emojis by typing "em emojiname".

The problem with the Fn key is that on most PC keyboards/laptops it is a key that is directly handled by the BIOS of the machine, to activate special hardware/media function keys, or flip the functions of the function keys row (often depends on a BIOS setting with PC keyboards). If you run evtest on such a machine (disable Toshy to look at the output from the real keyboard device, or look at the output from the xwaykeyz virtual keyboard), the Linux input system literally can't see you pressing the Fn key. But it will see the modified key code that will come from the key that you pair with the Fn key as a combo.

On the other hand, if you happen to be on a Mac portable or using an external Apple keyboard, the keymapper can see the Fn key event. This is already used in a shortcut that can disable/enable the "Forced Numpad" feature with Fn+NumLock, exactly the same as using Fn+Clear on an Apple keyboard with Numpad. So it is possible to remap the Fn key, but it will generally only work on an actual Apple keyboard. Possibly some PC keyboards that use the hid_apple driver to support extra media/hardware functions like an Apple keyboard.

I can imagine a popup like that is not standardized across DEs, and that is probably the reason why this feature hasn't been planned. But still, I would like to hear if this was considered at some point.

True, it doesn't even seem to be standardized across apps on a GNOME desktop. But, since there is an existing keyboard shortcut, at least in GNOME, that will open an emoji dialog at least some of the time, this could be part of the keymaps that are specific to certain desktop environments. If it exists on GNOME it may also have a counterpart on some other DEs. I'll have to look into that.

For me, I might even be able to redirect the remap to activate Ulauncher and type "em ", but since I'm on a PC laptop I won't have access to the Fn key, so I don't know what shortcut I could use to make it any more convenient than doing Cmd+Space and typing "em " myself.

I can show you how to do a custom user remap of Fn to Ctrl+Semicolon, but if you're not using an Apple keyboard you'd have to pick some other shortcut for the input side of the remap, and you can't just choose to tap a modifier key (Shift, Ctrl, Alt, or Meta/Super/Win/Cmd) as a shortcut by itself, I don't think the keymapper likes that. It wants to see either regular non-modifier keys or "combos" as inputs. So if you aren't on an Apple keyboard the exercise would be sort of pointless. You'd just be replacing a convenient combo with some other non-standardized combo shortcut.

In short, the practicality of trying to have a shortcut for this to match real Macs is quite limited. And then there's the fact that Fn is conveniently on the bottom left corner of Apple keyboards, but the Ctrl key is in that position on most PC keyboards, with Fn to the right of it. So the muscle memory factor is also limited.

Here's how you would do the custom remap if you happen to be on an Apple keyboard. Find the user_apps editable "slice" in your config file, and add a keymap like this (the "User overrides: GNOME" keymap):

###################################################################################################
###  SLICE_MARK_START: user_apps  ###  EDITS OUTSIDE THESE MARKS WILL BE LOST ON UPGRADE

keymap("User hardware keys", {

}, when = matchProps(not_lst=remotes_lod))

keymap("User overrides: GNOME", {
    C("Fn"): C("C-Semicolon"),      # Custom remap for Fn to open emoji picker in GNOME
}, when = lambda ctx: 
        matchProps(not_lst=remotes_lod)(ctx) and 
        DESKTOP_ENV == 'gnome'
)

###  SLICE_MARK_END: user_apps  ###  EDITS OUTSIDE THESE MARKS WILL BE LOST ON UPGRADE
###################################################################################################

You can open the Toshy config folder from the tray icon menu, it's in the normally hidden folder ~/.config/toshy/. The file is toshy_config.py. Save the changes are restart Toshy services from the tray icon menu, or in the terminal with toshy-services-restart. The new remap should become active.

Alberto-vgdd commented 1 month ago

Hi, thanks for the complete answer, I think that explains all of the questions I had about the request. It is now clear to me that the issue is even more complex than I anticipated at first. I will stick to the custom shortcut you suggested for now, as it will do the trick for me.

I am having issues using it, though. Whenever I add that bit of code to my toshy_config.py and then restart the changes none of the functionalities of Toshy work. I have to copy the contents of the default file from github in order to make it work again. Is the code snipped correct?

RedBearAK commented 1 month ago

I am having issues using it, though. Whenever I add that bit of code to my toshy_config.py and then restart the changes none of the functionalities of Toshy work. I have to copy the contents of the default file from github in order to make it work again. Is the code snipped correct?

I can't see anything wrong with it. I copied the keymap from my own customized user_apps section and just put different stuff inside it.

You don't need to copy the default file from GitHub, I keep a copy of the default config in a folder in that same Toshy config folder, for exactly this kind of scenario. See inside the "default-toshy-config" folder. Make a copy of the file, don't move it.

But before that, use toshy-config-verbose-start in the terminal to see debugging output that should immediately reveal the problem with the config, after you make the change. I don't think the keymapper designates Fn as a modifier, so it shouldn't be bothered by just having Fn alone in the input combo. I don't know what else could be wrong without seeing the error, if there is one. Copy and paste the initial part of the log in the terminal, after pressing a single key.

RedBearAK commented 1 month ago

Gah! My mistake for not running it to test, but I really didn't expect to see a KeyError when this other line with "Fn" has never caused a problem:

    C("Fn-Numlock"):            toggle_forced_numpad,           # Turn the Forced Numpad feature on and off

I'll have to look into why, but the immediate fix should be to replace "Fn"" with "KEY_FN"" or "Key_Fn", since that's the way the Fn key appears to be defined. Never ran into this problem before. (Edit: There's a modifier alias, so all this is wrong.)

    KEY_FN     = 0x1d0

There's a filter that removes the need to use the KEY_ prefix for a lot of keys, but maybe it only gets applied to regular letters and numbers. That still doesn't explain why the combo that's been in the config file for a year has never caused this exception. (Edit: Actually that's only applied to numbers, but letters are naturally defined without the KEY_ prefix in the keymapper's key definition file. So I guess that other remap was always wrong, it just doesn't cause the immediate error.)

Very odd.

RedBearAK commented 1 month ago

OK, never mind the edit to the last comment. At this point I'm very confused about what's happening with this key.

Traceback (while executing your config):
  File "/home/kris/.config/toshy/toshy_config.py", line 4301, in <module>
  File "/home/kris/.config/toshy/.venv/lib64/python3.12/site-packages/xwaykeyz/config_api.py", line 347, in combo
key = Key[key_str]
  File "/home/kris/.config/toshy/.venv/lib64/python3.12/site-packages/xwaykeyz/models/key.py", line 16, in __getitem__
return s.__getitem__(key)
  File "/usr/lib64/python3.12/enum.py", line 814, in __getitem__
return cls._member_map_[name]
KeyError: 'KEY_FN-NUMLOCK'

That makes no sense. There's no error when the single key is named "KEY_FN", but there is an error when the same key in the combo is named the same way.

This is not the usual kind of problem that occurs when trying to use a new remap. Usually the culprit that causes the config loading to fail is just a misplaced comma or missing closing parentheses or brace.

And at this point I'm not even confident that the C("Key_Fn") will actually work, even if you're on an Apple keyboard. I have an external USB Apple keyboard, but unfortunately it is French AZERTY and doesn't appear to have an Fn key anywhere, even in the center cluster. So I can't test the suggested remap at all, unless I bring out one of my MacBooks.

Alberto-vgdd commented 1 month ago

image Screenshot from 2024-05-16 23-38-25

I edited my config from what I understood from your last comment, and also ran the console command to check the output. The above screenshot shows the output when I press the "fn" key on my macbook. The mapping doesn't seem to work, but I couldn't spot any errors on the console. Am I missing something?

Here is the complete output until I press fn for the first time:

➜  ~ toshy-config-verbose-start

Stopping Toshy systemd services...

Toshy systemd services stopped.

xwaykeyz v1.0.1
(DD) CONFIG: /home/alberto/.config/toshy/toshy_config.py
(DD) THROTTLES: Pre-key: 12ms, Post-key: 18ms
(DD) No Synergy log folder found. No log observer will be engaged.

(CG) Current settings:
        -------------------------------------------
        calling_module          = 'toshy_config.py'
        prefs_db_file_path      = '/home/alberto/.config/toshy/toshy_user_preferences.sqlite'
        -------------------------------------------
        autostart_tray_icon     = True
        gui_dark_theme          = True
        -------------------------------------------
        override_kbtype         = 'Auto-Adapt'
        -------------------------------------------
        optspec_layout          = 'US'
        -------------------------------------------
        forced_numpad           = True
        media_arrows_fix        = False
        multi_lang              = False
        Caps2Cmd                = False
        Caps2Esc_Cmd            = False
        Enter2Ent_Cmd           = False
        ST3_in_VSCode           = False
        -------------------------------------------

(CG) Toshy config sees this environment:
    DISTRO_ID        = 'fedora'
    DISTRO_VER       = '40'
    SESSION_TYPE     = 'wayland'
    DESKTOP_ENV      = 'gnome'
    DE_MAJ_VER       = '46'

(DD) ENVIRON: Session type: 'wayland', Desktop env: 'gnome'
(DD) Zenity command path: '/usr/bin/zenity'
(--) WATCH: Watching for new devices to hot-plug.
(--) Autodetecting all keyboards (--device not specified)
(+K) Grabbing 'Apple Internal Keyboard / Trackpad' (/dev/input/event1)
(+K) Successfully grabbed 'Apple Internal Keyboard / Trackpad' (/dev/input/event1)
(--) Ready to process input.

(II) in KEY_FN (press)
(EE) D-Bus error querying GNOME Shell extension 'xremap@k0kubun.com':
    org.freedesktop.DBus.Error.UnknownMethod: 
     Object does not exist at path “/com/k0kubun/Xremap”
(EE) D-Bus error querying GNOME Shell extension 'window-calls-extended@hseliger.eu':
    org.freedesktop.DBus.Error.UnknownMethod: 
     Object does not exist at path “/org/gnome/Shell/Extensions/WindowsExt”
(DD) SHELL_EXT: Using UUID 'focused-window-dbus@flexagoon.com' for window context
(DD) on_key KEY_FN press
(DD) suspending keys: [Fn<Key.KEY_FN>]

(II) in KEY_FN (release)
(DD) SHELL_EXT: Using UUID 'focused-window-dbus@flexagoon.com' for window context
(DD) on_key KEY_FN release
(DD) resume because of mod release
(DD) resuming keys: [<Key.KEY_FN: 464>]
(OO) press KEY_FN 1715895733.6218526
(OO) release KEY_FN 1715895733.6219532
RedBearAK commented 1 month ago

OK, you're having better luck than me, but here is the problem:

# Fn is either invisible to the OS (on some laptop hardware) or it's just a
# normal key, but as a normal key it likely should be flagged as a modifier
# based on how it's typically used
Modifier("FN", aliases=["Fn"], key=Key.KEY_FN)

It is indeed classed as a modifier inside the keymapper, with an alias of "Fn", which was why the combo was not causing the exception. So this is simply a non-starter, as long as Fn is treated as a modifier key. And it makes sense to treat it as a modifier, at least on a keyboard where it can be seen by the keymapper.

Like I said earlier, the keymapper doesn't want to see modifier keys being used by themselves. That has a tendency to cause interference with actually using the same keys as parts of combos. And causes the exception I was seeing.

Now, the key could be converted to another innocuous key with a "multipurpose" modmap, so that if you tapped it it would be one thing, and if you held it and combined it with some other keys it would still be Fn. But it would have to be modmapped to something like an obscure function key that nobody uses, like F18 or something. Otherwise there would be no safe way to do a general remap of that resulting single key onto the combo that invokes the emoji picker. You wouldn't want to be having a general remap of the "A" key, for instance.

So this seems even more impractical than I was imagining. Multipurpose modmaps can be unreliable, due to some glitch deep in the keymapper logic that nobody has been able to identify.

Sorry about getting your hopes up. We can still try the multipurpose modmap if you've got the patience for it. It would probably work.

RedBearAK commented 1 month ago

Here's the multipurpose modmap.

###################################################################################################
###  SLICE_MARK_START: user_custom_modmaps  ###  EDITS OUTSIDE THESE MARKS WILL BE LOST ON UPGRADE

multipurpose_modmap("User override: Fn is F23 or Fn", {
    Key.KEY_FN:                  [Key.F23, Key.KEY_FN]     # Enter2Cmd
}, when = lambda ctx:
    matchProps(not_lst=remotes_lod)(ctx) )

###  SLICE_MARK_END: user_custom_modmaps  ###  EDITS OUTSIDE THESE MARKS WILL BE LOST ON UPGRADE
###################################################################################################

(Note that this is in a different place in the config file, a slice marked with user_custom_modmaps.)

Then you'd need to cause your custom override to remap C("F23") to Ctrl+Semicolon, instead of trying to remap the Fn modifier. A tap of the Fn key should work, but holding it as part of a combo will still work for other things.

In theory.

Alberto-vgdd commented 1 month ago

Alright, I followed your instructions and I think we are getting close to the intended behaviour. This is what happens at the moment when I press the "fn" key:

https://github.com/RedBearAK/toshy/assets/22345650/fd61e7f5-1d90-4397-a2ea-14a29f680cd0

The pop up appears as expected, but I also get this funky display in the middle of the screen, which I imagine means "F23 not mapped" in GNOME, which leads me to belive the F23 keypress is still being sent along with the shortcut

RedBearAK commented 1 month ago

Wow, it's working. OK, so the F23 keystroke does get "seen", I'm pretty sure all the input keystrokes actually get seen, which is weird, but that's why when a user shortens the "suspend" timeout of 1 second, we have to implement some annoying fixes of certain apps like Firefox and VSCode, to keep them from focusing the menu bar when they see Alt being pressed. That's just sort of a fundamental problem we have to deal with.

What the OSD probably means is that F23 is actually mapped to something. Let's see... Yeah, looks like that icon is showing the trackpad being disabled. So several of the higher function keys are actually mapped to things:

https://www.reddit.com/r/linuxquestions/comments/r9w8yh/disable_function_keys_beyond_f12/

So, maybe this is why the config file already uses F19 in an attempt to trick VSCode into not activating the menu for certain shortcut remaps. That's happening on the output side of the remaps, so it should be safe to just re-use F19 in the multipurpose modmap.

Don't forget to change your custom remap to match.

RedBearAK commented 1 month ago

BTW, I hope that F23 didn't actually disable your trackpad, if you have one, or you'll need to get something to emit the F22 key code if you don't have those physical keys on your keyboard device. 😆

Are you on a MacBook of some kind or just using an external Apple keyboard?

Alberto-vgdd commented 1 month ago

👌👌👋✌️🤙

It is working fine now, don't worry, F23 didn't disable my trackpad. I am using an actual Macbook, that's why I really wanted the Fn key to work.

I ended up mapping the key to F19 and it seems to behave fine just now, so that is great news.

Alberto-vgdd commented 1 month ago

I will set the issue as closed, hopefully if someone needs it in the future this will be of help.

RedBearAK commented 1 month ago

I ended up mapping the key to F19 and it seems to behave fine just now, so that is great news.

@Alberto-vgdd

Fantastic. The fact that you were able to survive this buggy process of figuring out how to get it to work is extraordinary.

I'm not sure I'll be adding the multipurpose modmap and a remap to the default config. Have to think about whether that could be detrimental. Maybe.

Meanwhile I can document this in a simpler way in the Wiki at some point.

Thanks for the interesting issue.

Alberto-vgdd commented 1 month ago

Thank you for the quick responses and support

RedBearAK commented 1 month ago

Did a write-up and added it to the Wiki:

https://github.com/RedBearAK/toshy/wiki/Using-Fn-key-for-emoji-picker

RedBearAK commented 1 month ago

@Alberto-vgdd

https://github.com/RedBearAK/toshy/wiki/Using-Fn-key-for-emoji-picker

Fleshed out the Wiki page with descriptions of how to handle this in a mixed GTK/Qt environment, including how to make a list of specific GTK apps to apply the remap to, in a way that would work in a non-GNOME environment.

I think that's probably the way to go ultimately, because the emoji picker is most likely built into the GTK framework itself, not the GNOME desktop environment. So having a global shortcut for it in "GNOME" may not make as much sense.

KDE has their own app (Emoji Selector), but it's literally just an app, like the old Character Palette window. The GTK emoji picker seems to be more like a virtual keyboard, like in macOS. That's why it doesn't work in other apps even while I'm in GNOME.

Alberto-vgdd commented 1 month ago

The wiki seems very complete and on point with the steps I followed. I think if someone else if facing the same issue, they will be able to follow it on their own. Thanks for the addition 😄