SDL2 lacks a way to get the real key that was pressed

[SDLBugzilla]

This bug report was migrated from our old Bugzilla tracker.

Reported in version: 2.0.10 Reported for operating system, platform: All, All

Comments on the original bug report:

On 2020-02-22 06:50:37 +0000, Unarelith wrote:

On QWERTY keyboard, slash ('/') key is a physical key, so it has both a keycode and a scancode, and can be mapped into a game easily.

On AZERTY keyboard, slash key is the combination of colon (':') and shift key.

This causes some problems:

• It has no valid scancode on AZERTY
• It's impossible to detect it using keycodes, since event.key.keysym.sym will be SDLK_COLON in that case

SDL1 used to have event.key.keysym.unicode which could be converted to SDL_Keycode.

SDL2 doesn't provide a way to transform SDLK_COLON + shift modifier to SDLK_SLASH, making slash key mapping dependent on keyboard layout.

For example, if I say that SDLK_SLASH is the "command" key, it will work on QWERTY, but not on AZERTY. If I say that SDLK_COLON + Shift (so basically: '/') is the "command" key, it will work on AZERTY but not on QWERTY.

So the problem is, SDL2 doesn't provide a way to know which key was actually pressed, even though SDL1 did.

It's still possible to use SDL_TEXTINPUT events to grab the keys and to convert them to SDL_Keycode, but this causes some issues:

• There will be three events, one for SDL_KEYDOWN, one for SDL_TEXTINPUT and one for SDL_KEYUP
• SDL_TEXTINPUT uses a string instead of just one char

So basically it's usable, it's just that the code needed to make SDL_TEXTINPUT working as a replacement of SDL_KEYDOWN in some specific cases, is not really worth it.

What I'd suggest to do in SDL2:

• Add back 'event.key.keysym.unicode' field, but add a warning in the documentation that it's not supposed to be used for getting text input, because that's what SDL_TEXTINPUT is for

OR

• Provide a new field, for example 'event.key.keysym.realsym' which will be equal to the keycode of the key that's actually pressed. So for 'Shift+:' on AZERTY, it would give SDLK_SLASH.

On 2020-03-02 02:12:43 +0000, Sam Lantinga wrote:

It sounds like you are trying to use keycodes as though they are scancodes. The scancodes are physical keys on the keyboard. Keycodes are the unshifted characters you get when the keys are pressed. The text events are the actual characters generated by the keyboard state when keys are pressed.

As you noted SDLK_SLASH may not exist on a keyboard, and you should never bind anything to it. You should instead bind to the scancode at that location, and when you're displaying the bindings, show the keycode that is in the current mapping.

On 2020-03-07 13:18:57 +0000, Unarelith wrote:

Sounds like you're misunderstanding me.

I'm looking for a way to actually recognize '/' key no matter what combination leads to it.

For an AZERTY keyboard, '/' key is SDLK_COLON + Shift. For a QWERTY keyboard, '/' key is SDLK_SLASH. For other keyboards, it may be another combination of a keycode + Shift.

SDL1 had the 'unicode' field that allowed recognizing SDLK_COLON + Shift as '/'.

SDL2 only has text events for that but they're really not practical. They give a string, and there's two events: one for the key press and one for the text event. This makes something like this waaaaay harder.

Adding back the 'unicode' field to SDL2 would be a great way to fix this issue actually. I don't know if that's possible or not though.

On 2020-10-21 00:39:12 +0000, H. Peter Anvin wrote:

So I would like to add my 2¢ to this.

I maintain a retro computer simulator. I very much do not want composition handling, dead keys, at so on.

What I would really like is to be able to answer the question: for scan code X, what are the symbols on the keycaps? For example, a Swedish keyboard has "¨^" where a U.S. keyboard has "] }". However, the former is a dead key; it will not, by itself, generate any text input event, and correctly so. But that is not my aim. SDL1 kind of happens to do "almost" the right thing, but what would be far better would be to be able to get all the keycaps printed (or normally printed) on a certain key; with a strong preference for having them in order of shift level (so a Swedish keyboard would report "+?\" for SDL_SCANCODE_MINUS.

That way I can produce a meaningful keyboard map that is a reasonable mapping onto the legacy computer keyboard on startup. For example, if I can see there is a key "[ {" like on a U.S. keyboard, I probably want to treat it as "{ [" and make it sensitive to Caps Lock. (ISO 646 fun.) This is really only possible if the whole keymap can be queried in bulk.

[williamhCode]

I would like to give my two cents on this issue. I'm currently working on a gui for Neovim and I have to send key sequence strings like j or <C-*>. But the problem is there's no real way of determining the string I should send due to differences in layouts. I would have to manually see if shift is being held, and translate the key names to the corresponding chars (which is keybord dependent).

If I use the text input event however, it doesn't process any keys that do not produce text and gets affected by deadkeys + special text combinations (e.g. option + key on macOS). It would be really convenient to add something like keysym.text or keysym.realsym discussed above, so I could get the "logical key" pressed and process the key event all in one place. Then I could use text input separately when I'm actually editing text (but still be able to use key event if I want to ignore deadkeys and ime).

I think an alternative better solution could be changing SDL_GetKeyName like this

SDL_GetKeyName(SDL_Keycode key, SDL_Keymod mods);

so users can get the "logical key" while deciding which modifiers they want to apply.

Overall I think this change would be pretty important to anyone making text editors and terminals, and is frankly what many windowing libraries overlook (personally I think winit does a very good job at this https://docs.rs/winit/latest/winit/event/struct.KeyEvent.html).

The discussion here also goes more in detail to different layouts that cause problems: https://github.com/glfw/glfw/issues/1618

[slouken]

@williamhCode, can you explain a bit more? I understand that if you want 'j' to move down, you'd look for SDLK_j, and if you want 'ZZ' to save and quit, you'd look for SDL_KMOD_SHIFT+SDLK_z, but you don't know that you want to look for that, because you don't know what combination of keys would generate 'Z'? Is that right?

It sounds like vim just wants to interpret the text, but that would break Ctrl+[key], because you would get a translated character as a result.

SDL aside, how would this normally be handled if you were using the native windowing system? It seems like you'd have the same problem?

[williamhCode]

@williamhCode, can you explain a bit more? I understand that if you want 'j' to move down, you'd look for SDLK_j, and if you want 'ZZ' to save and quit, you'd look for SDL_KMOD_SHIFT+SDLK_z, but you don't know that you want to look for that, because you don't know what combination of keys would generate 'Z'? Is that right?

Yes precisely. It seems like you do know a fair bit of vim yourself with the ZZ shortcut :)

It sounds like vim just wants to interpret the text, but that would break Ctrl+[key], because you would get a translated character as a result.

To map modifiers in vim, you do <C-char> for Ctrl, <M-char> for meta/alt, and <D-char> for cmd/super, <S-char> for shift. So i basically have to translate keyboard input to the correct text that is mapped in vim.

Well something like shift+z -> Z is pretty standard, and I would have no problems translating it manually. If I send <S-z>, that will even work as vim interprets that as Z. But for things like { and +, I can't reliably know when they're invoked. I don't want to force users to map <M-+> as like <S-M-=>, as that is keyboard independent, and would break in the terminal and other guis too.

SDL aside, how would this normally be handled if you were using the native windowing system? It seems like you'd have the same problem?

Here's a simple cpp program for macos, to get a character from a keycode + mods: https://gist.github.com/williamhCode/e5e6f76931a6f0df46f7c1a23376586a

This is basically equivalent to something like SDL_GetKeyName(SDL_Keycode key, SDL_Keymod mods); that i mentioned previously.

So for something like ctrl+cmd+shift+5, which I want to translate to <C-D-%>, assuming I have the function above, I can discard SDL_KMOD_CTRL and SDL_KMOD_GUI, call the function with the key (SDLK_5) and remaining modifiers (SDL_KMOD_SHIFT), and I'll get % and then I can combine it with the discarded modifiers to get <C-D-%>. This even works with something like sending <C-[> on the standard german keyboard, where option+8 -> [, so users can press ctrl+option+8 to send ctrl+[.

If the user, however, wants to use the option key as meta, instead of producing special characters, I can additionally discard SDL_KMOD_LALT or SDL_KMOD_RALT or both, and option+a will send <M-a> instead of å.

[grorp]

At Minetest, we also have a use case for this feature request. Minetest has a chat console where the player can enter commands. Commands start with '/'. While the player is in-game, we want to automatically open the chat console when the player presses '/'.

With Irrlicht, this was easy to implement because Irrlicht provides a char for each keyboard event.

We're currently in the process of migrating Minetest's Irrlicht fork to use SDL under the hood. Unfortunately, it's impossible to implement this properly with SDL as far as I can see. '/' is mapped to different key combinations on different keyboard layouts and there is no way to get a char from an SDL_KeyboardEvent.

We can't use SDL_TextInputEvent to listen for '/' while in-game either because it's only sent when text input is enabled. Enabling text input, either by manually calling SDL_StartTextInput when in-game or by never calling SDL_StartTextInput/SDL_StopTextInput, breaks Minetest's controls when an IME is involved because keypresses no longer produce SDL_KeyboardEvent.

[slouken]

@icculus and I discussed it, and we're planning to change keysym.sym to be the translated character based on current modifier state. We'll also add a function that will return the keycode for a keysym that takes modifier state as a parameter. This will be distinct from text input events, which will handle IME and virtual keyboard input, but will be much closer to what people have been asking for.

[slouken]

The latest code in git works the way you'd expect, the keycode is now upper case when the shift key is held down. This is implemented for Windows and macOS, and @Kontrabant is working on the Linux implementation.

Please try it out and let me know how it works for you!

[slouken]

Here's the relevant migration documentation: https://github.com/libsdl-org/SDL/blob/ba188e7555b873cbb57c9c5ee7ae6dc5a7bd872e/docs/README-migration.md?plain=1#L350-L365

[grorp]

Thanks for these changes. They work for me in Minetest and it's now quite possible to get chars from SDL key events (https://github.com/grorp/minetest/compare/ddb7bbecd8810e78c7a1775f4fda90a28f12ba2e...e07650956db93c91b8afd9ff3bb4f466b2d70fe5).

Since the keycodes for numpad keys are not the same as the keycodes for the equivalent non-numpad keys and don't correspond to chars, I still have to do some hackery to also get correct chars for the numpad +/- keys (https://github.com/grorp/minetest/commit/2205086cec91beb2f5ce727f8210fb86789e2ceb, comments not quite correct). I guess this is not something that needs a change in SDL and the use case is quite unusual, but I still wanted to mention it since it's not as straightforward as I had hoped.

While it doesn't matter for Minetest since we need both the unmodified and the modified keycode, I'm not sure whether event.key.key being affected by modifiers by default is the best approach. I imagine having to check for both lowercase and uppercase variants of letters keycodes could get quite annoying. However, this is only speculation.

[slouken]

The jury is still out on whether modified keycodes or unmodified keycodes should be the default, but thank you for confirming that it works well for you!

I'm going to open a separate issue for the numpad keycodes. I can see an argument for it to be either way.

[williamhCode]

I tested out the latest changes and they're working great, SDL_GetKeyFromScancode is just what I needed for my convoluted vim key translation uses. Thanks for the quick changes!

However, I noticed that SDL_GetKeyName returns the incorrect string for lowercase letters. So like SDLK_a returns A instead of a, which I guess is because previously it wasn't possible to have lowercase letters.

[slouken]

I tested out the latest changes and they're working great, SDL_GetKeyFromScancode is just what I needed for my convoluted vim key translation uses. Thanks for the quick changes!

You're welcome!

However, I noticed that SDL_GetKeyName returns the incorrect string for lowercase letters. So like SDLK_a returns A instead of a, which I guess is because previously it wasn't possible to have lowercase letters.

When you're working with both upper case and lower case keycodes, you probably want to do something like this:

if (keycode & SDLK_SCANCODE_MASK) {
    return SDL_GetKeyName(keycode);
} else {
    return UCS4toUTF8(keycode);
}

@icculus, we should export SDL_UCS4ToUTF8() so people don't have to write this themselves!

[slouken]

Since the keycodes for numpad keys are not the same as the keycodes for the equivalent non-numpad keys and don't correspond to chars, I still have to do some hackery to also get correct chars for the numpad +/- keys (grorp/minetest@2205086, comments not quite correct). I guess this is not something that needs a change in SDL and the use case is quite unusual, but I still wanted to mention it since it's not as straightforward as I had hoped.

It's not the default behavior, but I added a hint to convert the numpad keycodes to their non-numpad equivalents in https://github.com/libsdl-org/SDL/commit/ed9bbb2dca47671933f81fb2d118039a4595a02f

[williamhCode]

I recently updated to the latest commit, and I noticed if I use SDL_StartTextInput(), SDL stops sending key events if key compositions/ime ui related actions are being processed. It's usually the desired behavior, but for compositions like option + n, that triggers ˜ composition for english keyboard, there's no way to get the normal key events which I need if option key is interpreted as meta. Could it be possible to add such option (for example, disable left/right option and other modifiers for TextInput), or send the keyboard events anyways and add a flag to indicate it shouldn't be processed?

[rubenwardy]

Another Minetest dev here, is there a solution for SDL2? We can't depend on an unreleased version of SDL3, and won't be able to require SDL3 for a few years