Expose standardized locales with added defaults

[dbnicholson]

Describe the project you are working on

The Block Coding plugin.

Describe the problem or limitation you are having in your project

When trying to add translations, it seems that the OS locale was being handled in surprising ways that would cause the wrong translations to be chosen. I think I've figured out the solution to that, but it would have been nice to see more detail about how Godot has handling the input locale string.

Describe the feature / enhancement and how it helps to overcome the problem or limitation

TranslationServer::compare_locales uses locales standardized with additional defaults that make the results surprising. For example, comparing zh and zh_CN is reported as an exact match because both locales get converted to zh_Hans_CN. I ran into this while working on https://github.com/godotengine/godot/pull/98743.

It would be nice if these standardized locales with added default were exposed through TranslationServer::standardize_locale. In other words, add a second parameter add_defaults: bool = false. That would allow users to see how actually Godot treats partial or even seemingly fully specified locales.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

var os_locale = OS.get_locale()
var std_locale = TranslationServer.standardize_locale(os_locale, true)
print("%s: %s" % [os_locale, std_locale])

If this enhancement will not be used often, can it be worked around with a few lines of script?

Mostly it's informational. You could work around the issue of the wrong locale being set by building up some heuristics and calling TranslationServer.set_locale() for specific cases that you know would work incorrectly. However, even then it would be nice to see if TranslationServer.standardize_locale() changed when adding defaults. If there was no difference between the two cases when passing in the OS locale, then you would know that Godot was already going to do what you'd expect.

Is there a reason why this should be core and not an add-on in the asset library?

The purpose is to expose details about the core locale implementation. You can't do that in an add-on.

[bruvzg]

uses locales standardized with additional defaults that make the results surprising. For example, comparing zh and zh_CN is reported as an exact match because both locales get converted to zh_Hans_CN.

This is based on the way different OSes and Godot editor translation files name locales.

Practically, translation for zh without country code or script should not exist, it's too ambiguous (so it does not matter how it's matched). But some platforms use zh_CN and some zh_Hans, so comparing these two should have higher score then comparing zh_CN and zh_Hant for example. There's locale_scripts array specifically for cases like this.

[dbnicholson]

Practically, translation for zh without country code or script should not exist, it's too ambiguous (so it does not matter how it's matched). But some platforms use zh_CN and some zh_Hans, so comparing these two should have higher score then comparing zh_CN and zh_Hant for example. There's locale_scripts array specifically for cases like this.

I completely understand why it's done that way. Tons of locales are going to be ambiguous. What I'm asking is for Godot to expose the way that it handles that ambiguity since it's a black box right now. How would I know zh was treated as zh_Hans_CN without reading the Godot code or experimenting with compare_locales?

[timothyqiu]

Practically, translation for zh without country code or script should not exist

I'm not familiar with this area. I've only seen Wikipedia use zh so far. It allows mixing all kinds of variants together (as users editing different text segments may come from different regions and use different scripts). Turning on auto conversion changes the locale to a more specific one.

For game translations, I think using zh indeed doesn't make much practical sense.

[starnight]

Actually, zh has two major series: zh_TW (tradition Chinese) and zh_CN (simplified Chinese). And, I agree "translation for zh without country code or script should not exist, it's too ambiguous."