VkKeyScanW(0) is a constant during runtime independent of keyboard layout changes

[lhecker]

Environment

Windows build number: 10.0.19042.0

Steps to reproduce

1. Add the "US" keyboard layout within the "English (United States)" language
2. Add the "United Kingdom Extended" layout within the "English (United Kingdom)" language

3. Run the following code:

   #define NOMINMAX
   #include <Windows.h>
   #include <cstdio>
   
   int main() {
       while (true) {
           printf("0x%x\n", LOBYTE(VkKeyScanW(0)));
           Sleep(1000);
       }
       return 0;
   }

4. Change the keyboard layout using Win+Space or similar

Expected behavior

VkKeyScanW(0) prints 0x32 if the US and 0x40 if the UK layout is selected. The value changes during runtime if the layout is changed.

Actual behavior

VkKeyScanW(0) will continue to return its initial value and not change if the keyboard layout is changed during runtime. MapVirtualKeyW appears similarly affected. (I haven't tried to reproduce this yet though.)

This has far reaching implications due to the widespread use of these functions in this code base and manifests itself in key combinations either not working at all, or producing incorrect VT sequences. The only way to fix the issue is by restarting the application.

[zadjii-msft]

Yikes. Thanks for taking a look at all this!

[lhecker]

Cause

Thanks to the awesome work from the ReactOS authors the origin of this issue is easy to find... VkKeyScanW uses the keyboard layout defined in the current PTHREADINFO!

The kernel code looks somewhat like this (source):

SHORT VkKeyScanExW(WCHAR ch, HKL dwhkl) {
    return (SHORT)NtUserVkKeyScanEx(ch, dwhkl, TRUE);
}

SHORT VkKeyScanW(WCHAR ch) {
    return (SHORT)NtUserVkKeyScanEx(ch, 0, FALSE);
}

DWORD NtUserVkKeyScanEx(WCHAR wch, HKL dwhkl, BOOL bUsehKL) {
    PKL pKl = NULL;

    if (bUsehKL) {
        // Use given keyboard layout
        if (dwhkl)
            pKl = UserHklToKbl(dwhkl);
    } else {
        // Use thread keyboard layout
        pKl = ((PTHREADINFO)PsGetCurrentThreadWin32Thread())->KeyboardLayout;
    }

    // ...
}

GetKeyboardLayout(0) will also not update during runtime.

Solution

This kinda works:

WORD VkKeyScan(wchar_t ch) {
    const auto tid = GetWindowThreadProcessId(GetForegroundWindow(), 0);
    const auto hkl = GetKeyboardLayout(tid);
    return LOWORD(VkKeyScanExW(ch, hkl));
}

We could intercept WM_INPUTLANGCHANGE.

[skyline75489]

Ha now you can forget about ReactOS and see how it's implemented in actual Windows 😉 @lhecker

[tig]

Came here to report this in a slightly different manner.

Terminal.Gui app developers would like to be able to change the keyboard layout while the app is running. We use the Win32 API MapVirtualKey in our WindowsDriver to process keyboard input and pass it onto apps. Apps need access to the full range of key input, such as binding an app command to Ctrl-Oem1, which is Ctrl-ç on a Portuguese keyboard.

We've discovered that WT is always using the keyboard layout the process started with, ignoring WM_INPUTLANGCHANGE.

Here's the repo I was going to post in a new issue. Hopefully it helps y'all fix this:

1) Set the Keyboard layout to ENG (the key two keys to the left from Enter will be ;/: which is VK_OEM_1). 2) Start a WT session 3) Press the ;/: key

4) Press Shift-;:

5) Press Ctrl-; (WT strips ctrl off any Ctrl key that is not bound to an action):

6) Switch the keyboard layout to POR (Win-Space cycles through loaded layouts).

7) Press the VK_OEM_1 key (on POR this is the key labeled ç. As expected:

8) Press the Shift-VK_OEM_1 key. As expected:

9) Now, this is where it gets interesting. Press Ctrl-VK_OEM_1. This SHOULD print ç...

10) To prove this, start a fresh WT console and Press Ctrl-VK_OEM_1

I've tried using GetKeyboardLayout and it always reports the same layout that was active when the process started.

If anyone has a workaround we can implement until this issue is fixed, I'd love to hear it!

[tig]

Workaround:

#if !WT_ISSUE_8871_FIXED // https://github.com/microsoft/terminal/issues/8871
    /// <summary>
    /// Translates (maps) a virtual-key code into a scan code or character value, or translates a scan code into a virtual-key code.
    /// </summary>
    /// <param name="vk"></param>
    /// <param name="uMapType">
    /// If MAPVK_VK_TO_CHAR (2) - The uCode parameter is a virtual-key code and is translated into an unshifted
    /// character value in the low order word of the return value. 
    /// </param>
    /// <returns>An unshifted character value in the low order word of the return value. Dead keys (diacritics)
    /// are indicated by setting the top bit of the return value. If there is no translation,
    /// the function returns 0. See Remarks.</returns>
    [DllImport ("user32.dll", EntryPoint = "MapVirtualKeyExW", CharSet = CharSet.Unicode)]
    extern static uint MapVirtualKeyEx (VK vk, uint uMapType, IntPtr dwhkl);

    /// <summary>
    /// Retrieves the active input locale identifier (formerly called the keyboard layout).
    /// </summary>
    /// <param name="idThread">0 for current thread</param>
    /// <returns>The return value is the input locale identifier for the thread.
    /// The low word contains a Language Identifier for the input language
    /// and the high word contains a device handle to the physical layout of the keyboard.
    /// </returns>
    [DllImport ("user32.dll", EntryPoint = "GetKeyboardLayout", CharSet = CharSet.Unicode)]
    extern static IntPtr GetKeyboardLayout (IntPtr idThread);

    //[DllImport ("user32.dll", EntryPoint = "GetKeyboardLayoutNameW", CharSet = CharSet.Unicode)]
    //extern static uint GetKeyboardLayoutName (uint idThread);

    [DllImport ("user32.dll")]
    extern static IntPtr GetForegroundWindow ();

    [DllImport ("user32.dll")]
    extern static IntPtr GetWindowThreadProcessId (IntPtr hWnd, IntPtr ProcessId);

    uint MapVKtoChar (VK vk)
    {
        var tid = GetWindowThreadProcessId (GetForegroundWindow (), 0);
        var hkl = GetKeyboardLayout (tid);
        return MapVirtualKeyEx (vk, 2, hkl);
    }
#else
    /// <summary>
    /// Translates (maps) a virtual-key code into a scan code or character value, or translates a scan code into a virtual-key code.
    /// </summary>
    /// <param name="vk"></param>
    /// <param name="uMapType">
    /// If MAPVK_VK_TO_CHAR (2) - The uCode parameter is a virtual-key code and is translated into an unshifted
    /// character value in the low order word of the return value. 
    /// </param>
    /// <returns>An unshifted character value in the low order word of the return value. Dead keys (diacritics)
    /// are indicated by setting the top bit of the return value. If there is no translation,
    /// the function returns 0. See Remarks.</returns>
    [DllImport ("user32.dll", EntryPoint = "MapVirtualKeyW", CharSet = CharSet.Unicode)]
    extern static uint MapVirtualKey (VK vk, uint uMapType = 2);

    uint MapVKtoChar (VK vk) => MapVirtualKeyToCharEx (vk);
#endif