API for keyboard layouts

[dhardy]

In minimal form:

• Notification of change in the active keyboard layout (#1907)
• Ability to query a key's label for the active keyboard layout

In extended form:

• Add a handle/identifier for KeyboardLayout (i.e. the key mapping used by software)
• Ability to query the current KeyboardLayout
• (Possibly) ability to enumerate all known KeyboardLayouts
• Ability to query a key's label (or unmodified output) given a KeyboardLayout and scan-code (or whatever replaces scan code in the new API)
• (Possibly) ability to query UTF-8 character output of some key press with given modifiers

This is created as a tracking issue and search target, not an attempt to solve this or design an API.

Also related: #200, #1806.

[dhardy]

Note also: this concerns software layout only. There are some keyboards that allow you to adjust scan-code output for each physical key; in this case neither the OS nor usually any other software layer (outside the keyboard) knows what the layout is, and as such it cannot be queried.

[TimJentzsch]

Prior art: KeyboardLayoutMap in JavaScript

[kchibisov]

Ability to query a key's label (or unmodified output) given a KeyboardLayout and scan-code (or whatever replaces scan code in the new API)

I've done some research and all of that is possible and desired, though, part of it could be hidden inside the current event handling (as in how the logical key is represented in different layouts currently active in the keymap). This is how the bindings are working on linux; the all layouts are enumerated and the keysym`s from each layout are mapped to keycode, so you have layout agnostic bindings. I'll work on implementation of such stuff as a part of keyboard api change follow up, given that I have real use case for it and suffer from such issue in winit for a long time.

Maybe on macOS could be done something like that, but I'm not entirely sure, can't say about windows as well.

Also, not sure whether we'll need a direct information about the layout.

[kchibisov]

Assigned myself for xkb.

[dhardy]

I updated the initial post. I don't see much need for anything beyond the minimal (change notification + label for active layout). IIRC the other points were motivated by a quick examination of the X11 APIs.

[kchibisov]

The translation of Keysym to Keycode is what required on Linux for shortcuts, which is also part of the layout logic. So you could compare cyrillic С to English C and have the same ctrl + c binding for both of them without using a pre-hardcoded keycodes.

[daxpedda]

This can be done with a W3C proposed Web API, currently a working draft and implemented on Chromium based browsers.

Draft: https://wicg.github.io/keyboard-map MDN: https://developer.mozilla.org/en-US/docs/Web/API/Keyboard.

[maroider]

This can be done with a W3C proposed Web API

It's unlikely to ever be implemented by Safari due to privacy concerns.

[daxpedda]

Same for Firefox: https://mozilla.github.io/standards-positions/#keyboard-map.

[Imberflur]

I made a hacky mapping from KeyCodes to Keys + KeyLocations on linux/macos/windows here: https://github.com/Imberflur/winit/compare/bd890e69aa0ca9e2b3b6ef5b047a44f05102af91...Imberflur:winit:key_layout_hack. Not sure if mapping to Keys is the best way to get a label in the first place though :thinking: (although one advantage over an API that just returns a string label is being able to easily provide custom shorthand labels for certain keys).

This is mainly just an attempt to temporarily have a way to display key labels while waiting for a more proper implementation. There is no event for layout changes and no handle type for keyboard layouts. The mapping method is accessible via an extension trait tacked onto EventLoopWindowTarget. It is mostly inspired by the existing winit code for handling key events, especially the parts for generating key_without_modifiers.

[kchibisov]

I'm thinking of having a way to get latin_key for the given non-latin key, because you don't need these label stuff on latin layouts.

It's true that we can have KeyCode -> Key mapping, but the thing is that input must be on Key's and not KeyCode, really.

Also, if you try to refer to the actual printed labels on the keys, this is so arbitrary, since for example I could have a laptop with US keyboard, but without a RU legends, then I have RU layout in use right now, and you show me the char Ю, but I have no clue where the Ю actually is, because I can only type it while doing a touch typing, but not when I press a hotkey.

However if you've showed me a latin key(they are on all keyboards with legends), I'd be able to press it correctly.

[Imberflur]

but the thing is that input must be on Key's and not KeyCode, really

Why is this? My impression was that for game-like applications, using KeyCodes for keybindings can be better, and that storing configured bindings as KeyCodes can be useful since it keeps the same key position over layout switches. Although I'm certainly not entirely sure at this point.

The docs even mention:

The most prevalent use case for this is games. For example the default keys for the player to move around might be the W, A, S, and D keys on a US layout. The position of these keys is more important than their label, so they should map to Z, Q, S, and D on an “AZERTY” layout.

[kchibisov]

Why is this? My impression was that for game-like applications, using KeyCodes for keybindings can be better, and that storing configured bindings as KeyCodes can be useful since it keeps the same key position over layout switches. Although I'm certainly not entirely sure at this point.

I'm just coming from the point of using dvorak, given that in mostly each game I have to rebind everything, because they bind by keycode sometimes, so while it works on sw dvorak, it suddenly breaks on hw dvorak, but it's a niche case, because if you want to bind WASD.

Though, maybe native keycodes is better for games, because they want physical position based bindings, but in GUIs you never want something like that.