Wrong locale picked for System Language

[timothyqiu]

Pixelorama version: 0.8.1 stable

OS/device including version: Not graphics related

Issue description: My macOS has its language set to Simplified Chinese. If Pixelorama's language is set to System Language, it's actually Traditional Chinese that's picked.

System Language is achieved like this:

https://github.com/Orama-Interactive/Pixelorama/blob/d85efce73ad4751158461a1565002150a188b9dc/src/Preferences/HandleLanguages.gd#L76-L77

As per https://github.com/godotengine/godot/pull/40708, OS.get_locale is not stable across platforms. It returns zh-Hans-CN in my case, and TranslationServer.set_locale treats the string like zh, dropping the "Hans-CN" (Simplified Chinese) part.

Steps to reproduce: Start Pixelorama on macOS with Simplified Chinese locale.

Or manually try TranslationServer.set_locale("zh-Hans-CN").

[dasimonde]

On linux (PopOS), I get the same behavior.

TranslationServer.set_locale("zh-Hans-CN")
print(TranslationServer.get_locale())

My output is zh

[dasimonde]

In godotengine/godot/blob/master/core/translation.cpp are following localizations for chinese

"zh", //  Chinese
"zh_CN", //  Chinese (China)
"zh_HK", //  Chinese (Hong Kong)
"zh_SG", //  Chinese (Singapore)
"zh_TW", //  Chinese (Taiwan)

[OverloadedOrama]

I suppose this issue will get fixed once https://github.com/godotengine/godot/pull/40708 is merged into Godot, right?

[timothyqiu]

Yes and no I think.

https://github.com/godotengine/godot/pull/40708 is mainly documenting the format of locale returned by OS.get_locale. It may fix a similar issue on Windows (returning zh on Simplified Chinese). But on macOS, the optional -Hans / _Hans part still makes TranslationServer's locale zh.

[dasimonde]

I would say it is a issue with godot.

static const char *locale_list[] = {

    ...

    "zh", //  Chinese
    "zh_CN", //  Chinese (China)
    "zh_HK", //  Chinese (Hong Kong)
    "zh_SG", //  Chinese (Singapore)
    "zh_TW", //  Chinese (Taiwan)

    ...

    nullptr
};

bool TranslationServer::is_locale_valid(const String &p_locale) {
    const char **ptr = locale_list;

    while (*ptr) {
        if (*ptr == p_locale) {
            return true;
        }
        ptr++;
    }

    return false;
}

String TranslationServer::standardize_locale(const String &p_locale) {
    String univ_locale = p_locale.replace("-", "_");

    int idx = 0;
    while (locale_renames[idx][0] != nullptr) {
        if (locale_renames[idx][0] == univ_locale) {
            univ_locale = locale_renames[idx][1];
            break;
        }
        idx++;
    }

    return univ_locale;
}

String TranslationServer::get_language_code(const String &p_locale) {
    ERR_FAIL_COND_V_MSG(p_locale.length() < 2, p_locale, "Invalid locale '" + p_locale + "'.");

// 3. if p_locale == "zh_hans_CN" then
//     split is 2
    int split = p_locale.find("_");         

// This block is skipped                                                   
    if (split == -1) {
        split = p_locale.find("-");
    }
// This block is skipped
    if (split == -1) {
        return p_locale;
    }

// 4. p_locale.left(split) is "zh"
    return p_locale.left(split);
}

void TranslationServer::set_locale(const String &p_locale) {

 // 1. After standardize_locale, univ_locale is "zh_Hans_CN"
    String univ_locale = standardize_locale(p_locale);

// 2. "zh_Hans_CN" isn't in p_local_list
//     that's why !is_locale_valid(univ_locale) is equal true
    if (!is_locale_valid(univ_locale)) {

        String trimmed_locale = get_language_code(univ_locale);
        print_verbose(vformat("Unsupported locale '%s', falling back to '%s'.", p_locale, trimmed_locale));

        if (!is_locale_valid(trimmed_locale)) {
            ERR_PRINT(vformat("Unsupported locale '%s', falling back to 'en'.", trimmed_locale));
            locale = "en";
        } else {
            locale = trimmed_locale;
        }
    } else {
        locale = univ_locale;
    }

    if (OS::get_singleton()->get_main_loop()) {
        OS::get_singleton()->get_main_loop()->notification(MainLoop::NOTIFICATION_TRANSLATION_CHANGED);
    }

    ResourceLoader::reload_translation_remaps();
}

String TranslationServer::get_locale() const {
    return locale;
}

String String::replace(const String &p_key, const String &p_with) const {
    String new_string;
    int search_from = 0;
    int result = 0;

    while ((result = find(p_key, search_from)) >= 0) {
        new_string += substr(search_from, result - search_from);
        new_string += p_with;
        search_from = result + p_key.length();
    }

    if (search_from == 0) {
        return *this;
    }

    new_string += substr(search_from, length() - search_from);

    return new_string;
}

[Variable-ind]

does anyone still experience this issue?