position based key binding support

[wez]

Today, key assignments can be made either via the raw keycode, a platform-dependent code that has different values depending on the windowing environment being used, or via the processed unicode values that are obtained after the OS keyboard layout has been applied to the raw input.

It is desirable to be able to express key bindings in terms of the physical position of the key on a US qwerty keyboard. For example I might want to make SUPER+Q quit wezterm, and make it so that that Q key is the one I press even if I switch to a keyboard layout such as AZERTY where the Q and the A keys are switched. In the AZERTY layout the Q has become an A, but that physical position still binds to the key assignment.

I happen to know from my mechanical keyboard building hobby that the key positions have defined codes in the USB HID specification. Those get interpreted by the OS and are exposed to wezterms input layer in the form of the raw values that we already support.

The goal then is to allow using a configuration like:

return {
   keys = {
     {physical="Q", mods="CMD", action="QuitApplication"},
   },
}

Internally, we'd need:

• [x] An enum of physical key positions that we can plumb through to the config
• map raw keycode to physical position:
  • [x] X11
  • [x] Wayland
  • [x] macOS
  • [x] Windows
• [x] Adjust keyboard input mapper to handle this, likely by adding an additional matching phase
• [x] Decide the precedence/match order of physical, raw and key based assignments -> physical, then raw, then mapped
• [x] Decide whether the default key assignments should be re-configured using physical -> physical
• [x] Should the key assignment syntax default to physical rather than the current cooked value? -> physical

Open Questions:

• Should the default key assignments really be position based?
• Should we allow the user to select position or mapped variations of the default key assignments?
• Is there something better we could do for non-US layout users? (potentially tangential to the main thesis of this issue)

[wez]

Current main allows:

local wezterm = require 'wezterm'

return {
  debug_key_events = true,
  keys = {
    {key="phys:Q", action=wezterm.action{SendString="woot"}},
  },
}

The above will send "woot" to the pane whenever the key in same physical position as Q on an ANSI standard US keyboard is pressed.

[JeanMertz]

@wez question, I just updated to the latest nightly, noticed my key configurations no longer working when they are targeted at the CMD modifier key, and found this issue. I searched through the change log, but couldn't find anything related that could cause this behaviour. Could it be related to the latest changes proposed in this issue?

These commands:

    keys = {
        {key="k", mods="CMD", action=wezterm.action{ClearScrollback="ScrollbackAndViewport"}},
        {key="<", mods="CMD", action=wezterm.action{MoveTabRelative=-1}},
        {key=">", mods="CMD", action=wezterm.action{MoveTabRelative=1}},
        {key="m", mods="CMD", action="Nop"}
    }

All stopped working for me because they use the CMD modifier.

I also tried renaming the modifier to SUPER, but it didn't change the outcome.

[JeanMertz]

Another — perhaps related — issue is that this mapping no longer works:

{key="n", mods="CTRL|SHIFT", action="ToggleFullScreen"},

From the change log, I understand I can now use this instead:

{key="N", mods="CTRL", action="ToggleFullScreen"},

But neither work as expected. Previously, this would toggle Wezterm's full screen, but now it seems to behave as if I type CMD+N, that is, it opens a new Wezterm window.

[wez]

@JeanMertz can you open a new issue for what you're seeing? I'd like to capture more information about your setup so we can figure what's going on!

[wez]

Summarizing the changes since the prior release:

• You may now use phys:A to refer to the key in the same physical position as A on an ANSI US layout keyboard
• Formally split the internal KeyEvent into RawKeyEvent and KeyEvent, which are the pre- and post-mapped/composed key events, respectively
• RawKeyEvents are matched against your key assignments (+ defaults). If they match then the raw key event will not contribute to dead key or other IME compositions.
• If RawKeyEvent didn't match any assignments, the event is fed through dead keys and composition to produce a KeyEvent
• KeyEvent is matched against your key assignments. If it doesn't match any, it is passed through to the pane.
• The default key assignments have all (except for ALT-<number>, see #1542) been redefined in terms of physical positions on an ANSI US layout keyboard. For users of non-US layouts, some of them (notably those on brackets and punctuation keys) may now appear to have stopped working because they live on different keys compared to a prior release.
• The config loader will now assume that key="a" should be key="phys:A" so that your key assignments in your config will match based on the physical position of the key in a US ANSI layout.
  • If you had {key="N", mods="CMD", ..} it will now be treated as though you wrote {key="phys:N", mods="CMD", ...}, BUT! the old syntax would only match a shifted, uppercase N, whereas the rewritten form cannot represent the shift state in the key portion. If you had assignments in that style, you will need to explicitly add SHIFT into your mods for them.
• You can use key="mapped:a" to explicitly refer to the post-mapped version of the key instead.

[JeanMertz]

@wez FYI, I managed to solve my above two issues by changing the above-mentioned mappings to these:

        {key="phys:K", mods="CMD", action=wezterm.action{ClearScrollback="ScrollbackAndViewport"}},
        {key="<", mods="SHIFT|CMD", action=wezterm.action{MoveTabRelative=-1}},
        {key=">", mods="SHIFT|CMD", action=wezterm.action{MoveTabRelative=1}},
        {key="phys:M", mods="CMD", action="Nop"},
        {key="phys:N", mods="SHIFT|CTRL", action="ToggleFullScreen"},

I assume you'd still like this to be backward compatible as much as possible, but at least the new feature-set allowed me to get my existing shortcuts working again.

[wez]

@JeanMertz with the latest main (should be available as a mac nightly download once this job completes: https://github.com/wez/wezterm/runs/4694742839?check_suite_focus=true), you can rewrite that as:

        {key="K", mods="CMD", action=wezterm.action{ClearScrollback="ScrollbackAndViewport"}},
        {key="<", mods="SHIFT|CMD", action=wezterm.action{MoveTabRelative=-1}},
        {key=">", mods="SHIFT|CMD", action=wezterm.action{MoveTabRelative=1}},
        {key="M", mods="CMD", action="Nop"},
        {key="N", mods="SHIFT|CTRL", action="ToggleFullScreen"},

[Mattwmaster58]

This changed seems to have introduced a regression in specific circumstances with modifiers. I've only seen it happen when mods="CTRL|SHIFT" and key is a number. For example:

    {key="1", mods="CTRL", action=wezterm.action{SendString="woot"}},
    {key="1", mods="CTRL|SHIFT", action=wezterm.action{SendString="woot"}},

The first shortcut works, the second does not.

As a temporary workaround I've found that this does work:

    {key="phys:1", mods="CTRL|SHIFT", action=wezterm.action{SendString="woot"}},

If it matters I'm on a windows machine, on a keyboard with no numpad

 keyevent > key_event KeyEvent { key: LeftWindows, modifiers: SUPER, repeat_count: 1, key_is_down: true }
 keyevent > send to pane key=LeftWindows mods=SUPER
 keyevent > key_event RawKeyEvent { key: RawCode(17), modifiers: CTRL, phys_code: None, raw_code: 17, repeat_count: 1, key_is_down: true, handled: Handled(false) }
 keyevent > key_event RawKeyEvent { key: Physical(T), modifiers: CTRL, phys_code: Some(T), raw_code: 84, repeat_count: 1, key_is_down: true, handled: Handled(false) }
 keyevent > Physical(T) CTRL -> perform SendString("woot")
 keyevent > key_event RawKeyEvent { key: RawCode(17), modifiers: CTRL, phys_code: None, raw_code: 17, repeat_count: 1, key_is_down: true, handled: Handled(false) }
 keyevent > key_event RawKeyEvent { key: RawCode(16), modifiers: SHIFT | CTRL, phys_code: None, raw_code: 16, repeat_count: 1, key_is_down: true, handled: Handled(false) }
 keyevent > key_event RawKeyEvent { key: Physical(K1), modifiers: SHIFT | CTRL, phys_code: Some(K1), raw_code: 49, repeat_count: 1, key_is_down: true, handled: Handled(false) }
 keyevent > Physical(K1) SHIFT | CTRL -> perform ActivateTab(0)

[Mattwmaster58]

FWIW, whatever you changed in these last commits has things working for me again 👍

[github-actions[bot]]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.