[Run] Packaged apps only available via localized name

[htcfreek]

Environment

Windows build number: 10.0.19041.264
PowerToys version: 0.18.2
PowerToy module for which you are reporting the bug (if applicable): PowerToys Run

Problem

For packaged apps we not indexing the englisch name.

Details / Example

In start I can search for Rechner or calculator to open the calculator app. The app is shown as "Rechner (= German)". In PT Run the app can only be found by its localized name Rechner.

Screenshots

Related to https://github.com/microsoft/PowerToys/issues/3773

[crutkas]

@htcfreek does this still happen in 0.19?

[htcfreek]

Yes.

[crutkas]

now for the ultimate question, how do we get that data :)

[htcfreek]

now for the ultimate question, how do we get that data :)

Yes that's the BIG question.

Maybe from the appx manifest file in the app folder used for registering the package. 🤔🤷

[crutkas]

removing this as this is unknown how to get this data

[htcfreek]

@crutkas Should we add the "help-wanted" and the "tracker" label?

[crutkas]

@htcfreek are you changing your language while PT Run is up and running? #4147 shows it with it grabbing it.

3773 could be related here as well.

[htcfreek]

@htcfreek are you changing your language while PT Run is up and running? #4147 shows it with it grabbing it.

3773 could be related here as well.

No. I didn't change my lang. I only search for English and for German name.

BTW: My sys and app lang is German.

[mxrsoon]

@htcfreek are you changing your language while PT Run is up and running? #4147 shows it with it grabbing it.

3773 could be related here as well.

The screenshot in #4147 that shows it grabbing "Câmera" for the "camera" query is a mockup, used only to illustrate the expected behavior.

[htcfreek]

@crutkas I think we can't fix this. The native api method we have to use for getting the name has no parameter to influence the locale used. Instead it uses the windows display language and we can't manipulate this language property at runtime. (It's only changeable by the shell itself.)

Should we close this issue?

[gexgd0419]

I think that we can parse the resource (.pri) file to read the display name string in a specific language from it.

We can pass @{Microsoft.WindowsTerminal_1.15.3466.0_x64__8wekyb3d8bbwe?ms-resource://Microsoft.WindowsTerminal/Resources/AppStoreName} to SHLoadIndirectString() to get Windows Terminal's display name in current display language. Since SHLoadIndirectString() loads the string from the app package's .pri file, we can also pass @{C:\Program Files\WindowsApps\Microsoft.WindowsTerminal_1.15.3466.0_x64__8wekyb3d8bbwe\resources.pri?ms-resource://Microsoft.WindowsTerminal/Resources/AppStoreName} to it, replacing the package name with the path to the .pri file, which is usually the package install path + "\resources.pri".

Unfortunately, changing the preferred UI language of the current thread/process doesn't affect its behavior, and there's no documentation on how SHLoadIndirectString() loads the string from .pri file. But fortunately, there's a library in Windows App SDK called MRTCore that provides a way to read any given .pri file.

After building the DLL, I used the following C++ code to test it.

MrmManagerHandle hResMgr;
// load the .pri file given its path
hr = MrmCreateResourceManager(LR"(C:\Program Files\WindowsApps\Microsoft.WindowsTerminal_1.15.3466.0_x64__8wekyb3d8bbwe\resources.pri)", &hResMgr);

MrmContextHandle hResCtx;
hr = MrmCreateResourceContext(hResMgr, &hResCtx);
hr = MrmSetQualifier(hResCtx, L"language", L"en");  // the language of the result string

LPWSTR pStr;
// load string given its resource URI
hr = MrmLoadStringResourceFromResourceUri(hResMgr, hResCtx, L"ms-resource://Microsoft.WindowsTerminal/Resources/AppStoreName", &pStr);

// now the string is stored in pStr

// free allocated strings & handles
MrmFreeResource(pStr);
MrmDestroyResourceContext(hResCtx);
MrmDestroyResourceManager(hResMgr);

Then I successfully get the display name of Windows Terminal in different languages by changing the "language" qualifier value to "en" Windows Terminal, "zh-hant" Windows 終端機, "ja" Windows ターミナル, etc. Setting language to an empty string will return the English version (default?), and if the language is not set, it will return the Simplified Chinese version Windows 终端 which is the display language on my system.

So maybe we can use this to replace the call to SHLoadIndirectString().

[htcfreek]

@gexgd0419 Sounds interesting. Please have in mind that the Program plugin is written in C#. And to be aligned to Win32 programs we need both localized and English name. The goal would be to show it with localized path and name, but can search for English and localized name.

If you want to create a PR this would be awesome. If you have some questions about the plugin code feel free to ask me/us.

[gexgd0419]

Here's a C# version. It requires Microsoft.WindowsAppSDK.

var resMgr = new Microsoft.Windows.ApplicationModel.Resources.ResourceManager(@"C:\Program Files\WindowsApps\Microsoft.WindowsTerminal_1.18.10301.0_x64__8wekyb3d8bbwe\resources.pri");
var ctx = resMgr.CreateResourceContext();
ctx.QualifierValues["Language"] = "en"; // Specify a language. Skip this line to use the user's display language
var candidate = resMgr.MainResourceMap.TryGetValue("Resources/AppStoreName", ctx);
if (candidate != null) // if exists
    Console.WriteLine(candidate.ValueAsString);

[gexgd0419]

I also saw #3773 and #4637, which requested that PT Run find the correct result when the user types the search term in one language, e.g. English, but forgets to switch the keyboard layout, so it gets typed in another keyboard layout, e.g. Hebrew. Unfortunately, those issues were closed as "duplicates", so I have to mention them here.

As I commented here, supposedly, Windows Search does this by maintaining a text file that contains some commonly used aliases and typos. If your typo happens to be in the list, Windows Search will correct it; otherwise, it can't do wonders, either.

However, Windows does have some APIs that can translate between characters and key codes, and in different keyboard layouts. See the following:

• VkKeyScanExW: character -> virtual key code & shift states
• MapVirtualKeyExW: virtual key code -> character, or virtual key code <-> scan code
• ToUnicodeEx: virtual key code & keyboard state -> character

All of them accepts an HKL which represents a keyboard layout. Use GetKeyboardLayout to get the current layout, or LoadKeyboardLayoutW to load a layout.

Here's my example written in C#. Note that not every key can be mapped correctly using this method; for example, dead keys will not be mapped, because they are not supported by VkKeyScanEx. Characters that cannot be mapped will be copied.

internal class Program
{
    internal enum VKMapType : uint
    {
        VkToSc = 0, ScToVk, VkToChar, ScToVkEx, VkToScEx
    }

    [DllImport("user32", CharSet = CharSet.Unicode)]
    internal static extern ushort VkKeyScanEx(char ch, nuint hkl);

    [DllImport("user32", CharSet = CharSet.Unicode)]
    internal static extern uint MapVirtualKeyEx(uint code, VKMapType mapType, nuint hkl);

    [DllImport("user32", CharSet = CharSet.Unicode)]
    internal static extern int ToUnicodeEx(uint virtkey, uint scancode, byte[] lpKeyState, StringBuilder buff, int cchBuff, uint flags, nuint hkl);

    [DllImport("user32", CharSet = CharSet.Unicode)]
    internal static extern nuint LoadKeyboardLayout(string KLID, uint flags);

    [DllImport("user32")]
    internal static extern nuint GetKeyboardLayout(uint threadId = 0);

    [DllImport("user32")]
    internal static extern bool UnloadKeyboardLayout(nuint hkl);

    [DllImport("user32")]
    internal static extern int GetKeyboardLayoutList(int nBuff, [Out] nuint[]? lpList);

    internal const uint VK_SHIFT = 0x10u;
    internal const uint VK_CONTROL = 0x11u;
    internal const uint VK_MENU = 0x12u;
    internal const uint KLF_NOTELLSHELL = 0x80u;

    // Version that accepts two keyboard layout identifiers (or "input locale identifiers").
    // Use GetKeyboardLayout(), LoadKeyboardLayout(), or CultureInfo.KeyboardLayoutId to get them.
    static string MapKeyboardLayout(string content, nuint srcLayout, nuint dstLayout)
    {
        if (srcLayout == dstLayout)
            return content;

        StringBuilder result = new StringBuilder(content.Length), buff = new StringBuilder(8);
        byte[] keyboardState = new byte[256];

        foreach (char ch in content)
        {
            // map character to virtual key code & shift state
            ushort scanresult = VkKeyScanEx(ch, srcLayout);
            if (scanresult == 0xFFFF) // failed, append the original character
            {
                result.Append(ch);
                continue;
            }

            // low byte: virtual key code; high byte: shift state
            uint virtkey = scanresult & 0xFFu;
            uint shiftstate = (scanresult & 0xFF00u) >> 8;
            // set the keyboard state array
            keyboardState[VK_SHIFT] = (byte)((shiftstate & 1) != 0 ? 0x80 : 0);
            keyboardState[VK_CONTROL] = (byte)((shiftstate & 2) != 0 ? 0x80 : 0);
            keyboardState[VK_MENU] = (byte)((shiftstate & 4) != 0 ? 0x80 : 0);

            // map virtual key code & keyboard state to character
            uint scancode = MapVirtualKeyEx(virtkey, VKMapType.VkToSc, srcLayout);
            int convresult = ToUnicodeEx(virtkey, scancode, keyboardState, buff, 8, 4, dstLayout);
            if (convresult > 0)
            {
                // succeeded; convresult is the string length
                if (buff.Length >= convresult) // sometimes character can be zero even if convresult>0
                    result.Append(buff.ToString(0, convresult));
            }
            else
            {
                // failed or dead key
                result.Append(ch);
            }
        }

        return result.ToString();
    }

    // Versions that takes two CultureInfo and loads the keyboard layout itself.
    static string MapKeyboardLayout(string content, CultureInfo srcCulture, CultureInfo dstCulture)
    {
        // Get the list of currently loaded keyboard layouts.
        int layoutCount = GetKeyboardLayoutList(0, null);
        nuint[] layouts = new nuint[layoutCount];
        GetKeyboardLayoutList(layoutCount, layouts);

        // Load the keyboard layouts corresponding to the cultures.
        // If the keyboard layout has been loaded (for example, chosen by the user), LoadKeyboardLayout() only returns it.
        // If hasn't, LoadKeyboardLayout() also loads the specified layout into the system.
        // Make sure to unload the "newly-loaded" layouts afterwards, to avoid adding new items to the keyboard layout list.
        // LoadKeyboardLayout accepts a hexadecimal string of the language ID, e.g. "00000409"
        var srcLayout = LoadKeyboardLayout(srcCulture.KeyboardLayoutId.ToString("X08"), KLF_NOTELLSHELL);
        var dstLayout = LoadKeyboardLayout(dstCulture.KeyboardLayoutId.ToString("X08"), KLF_NOTELLSHELL);

        string result = MapKeyboardLayout(content, srcLayout, dstLayout);

        // Unload the layouts that was not loaded before
        if (!layouts.Contains(srcLayout))
            UnloadKeyboardLayout(srcLayout);
        if (!layouts.Contains(dstLayout))
            UnloadKeyboardLayout(dstLayout);

        return result;
    }

    // Maps from the current keyboard layout to English standard QWERTY layout.
    static string MapKeyboardLayoutToEnglish(string content)
    {
        // Get the list of currently loaded keyboard layouts.
        int layoutCount = GetKeyboardLayoutList(0, null);
        nuint[] layouts = new nuint[layoutCount];
        GetKeyboardLayoutList(layoutCount, layouts);

        var enLayout = LoadKeyboardLayout("00000409", KLF_NOTELLSHELL);
        var curLayout = GetKeyboardLayout(0);

        string result = MapKeyboardLayout(content, curLayout, enLayout);

        // Unload the English layout if it wasn't loaded
        if (!layouts.Contains(enLayout))
            UnloadKeyboardLayout(enLayout);

        return result;
    }

    // Maps from English standard QWERTY layout to the current keyboard layout.
    static string MapKeyboardLayoutFromEnglish(string content)
    {
        // Get the list of currently loaded keyboard layouts.
        int layoutCount = GetKeyboardLayoutList(0, null);
        nuint[] layouts = new nuint[layoutCount];
        GetKeyboardLayoutList(layoutCount, layouts);

        var enLayout = LoadKeyboardLayout("00000409", KLF_NOTELLSHELL);
        var curLayout = GetKeyboardLayout(0);

        string result = MapKeyboardLayout(content, enLayout, curLayout);

        // Unload the English layout if it wasn't loaded
        if (!layouts.Contains(enLayout))
            UnloadKeyboardLayout(enLayout);

        return result;
    }

    static void Main(string[] args)
    {
        Console.InputEncoding = Encoding.Unicode;
        Console.OutputEncoding = Encoding.Unicode;

        // What will appear on the screen if I type "calculator" but accidentally switched to Russian layout? ("сфдсгдфещк")
        string s1 = MapKeyboardLayout("calculator", CultureInfo.GetCultureInfo("en-US"), CultureInfo.GetCultureInfo("ru-RU"));
        Console.WriteLine(s1);

        // What's the correct Russian text that shows as "rfkmrekznjh" after typing with English layout? ("калькулятор")
        string s2 = MapKeyboardLayout("rfkmrekznjh", CultureInfo.GetCultureInfo("en-US"), CultureInfo.GetCultureInfo("ru-RU"));
        Console.WriteLine(s2);

        // What's the correct English text that shows as "Ьшскщыщае Еуфьы" after typing with Russian layout? ("Microsoft Teams")
        // Note that it can convert upper/lower cases correctly.
        string s3 = MapKeyboardLayout("Ьшскщыщае Еуфьы", CultureInfo.GetCultureInfo("ru-RU"), CultureInfo.GetCultureInfo("en-US"));
        Console.WriteLine(s3);

        // Let the user type some text.
        nuint layout = GetKeyboardLayout(0);
        int langid = (int)(layout & 0x3FF);
        var culture = CultureInfo.GetCultureInfo(langid);
        Console.WriteLine("Your keyboard layout: " + culture.EnglishName);
        Console.Write("Enter some text in your keyboard layout: ");
        string? s4 = Console.ReadLine();
        if (s4 != null)
            Console.WriteLine("Mapped to English QWERTY layout: " + MapKeyboardLayoutToEnglish(s4));
    }
}

[htcfreek]

@gexgd0419 I think these are two different things:

1. Finding appx packages with the inglish name too. (Not only with their localized one.)
2. Switching the keyboard layout used for PT Run.

~We should split them into different issues. Otherwise tracking work will be complicated.~ Your last comment regarding the keyboard thing better fits for #31125.

[gexgd0419]

I agree that it would be better if PT Run can just get the keyboard layout right in the first place.

However, #3773 suggests that it work both ways:

• English typed in Russian layout
• Russian typed in English layout

Product Launcher should support keyboard layout mapping for all system keyboard layouts, for example:

US Keyboard ⟶ Russian Keyboard Russian Keyboard ⟶ US Keyboard

Query "ьшскщыщае еуфьы" should display the results of "ьшскщыщае еуфьы" and "microsoft teams" queries. Query "rfkmrekznjh" should display the results of "калькулятор" and "rfkmrekznjh" queries.

[gexgd0419]

The author of #3773 said:

Important note: My locale is "English US" and the region is the United States. Preferred languages:

• English US
• Russian

So English is the display language, and Russian is in the preferred language list.

Should we support searching in preferred languages other than the display language?

[htcfreek]

Should we support searching in preferred languages other than the display language?

Currently PT Run supports only the display language and English for Win32 programs. For packaged apps should do the same. (Everything else would be a rare case.)

@gexgd0419 Do you like to work on one of the issues?