Translations

[asumagic]

I have discussed translation support a bit tonight with Fuzzle and decided to do some work to expand the translation support in KAG.

I believe it would bring some good advantages. It is easier for non-native people to understand (= potentially less retention). It also would allow making some official translations appear on Steam. It could be possible to translate the steam shop page into other languages as well, I can translate it to french if you want.

I've done some work tonight to extend the translation support in KAG, see the changes on my repo (forgot to take it on a branch, I don't have time fixing it up tonight).
Some changes were less trivial than others. As for help support, e.g., I've taken the easy way and made a somewhat conservative way to perform the translation automatically (some helps translates manually). I didn't extensively try it, though, but it looks decent so far.

Though, I have a few questions:

• Irrlicht GUI: Is Locales/zh_**.irrst supposed to work in Release? I didn't ever manage to make it spit out translated text (with a zh_FR.irrst file with the locale set as fr in autoconfig or with zh_EN.irrst with the locale set as en in autoconfig). I was writing it this way:

  <entry id="PLAY">JOUER</entry>
  <entry id="Create Account">Créer un compte</entry>

  It also contains some outdated strings, it would be nice to make sure everything is up to be translatable not to miss out some stuff.

• Is having partial translations OK? The polish translation seems to translate just a small batch of strings. I would rather progressively push up the newer translation support rather than all at once (to avoid some git merging nightmare considering how much scripts are affected). I suggest an option in the settings to select the locale and mark non-english as "experimental" support while fallbacking all current clients to EN.
• How possible could it be to have blob inventory names/etc translated by the engine? This would ease out the translation support for sure, and I doubt there are scripts that relies on the inventory name (hopefully...)
• Could it be possible to avoid ambiguous translations in the json file e.g. by allowing packing specific things in objects? For example in a case such as:

  
  "scoreboard":
  {
  "kills": "meurtres", // noun
  "deaths": "morts",
  // ...
  }

"killfeed": { "kills": "a tué" // verb }```

• Edit: it also might be a solution to use a whole different translation system maybe. It would be extra neat to have some way to way to format the text in the translations (e.g. {0} has killed {1}) since it might be complicated to translate some stuff with the same ordering as in english. I could probably implement it in angelscript directly though.

[1bardesign]

The XML stuff is deprecated and from classic.

The json-based cl_locale translations are "the way forward" though they aren't very fleshed out at the moment because there has been minimal community interest. There isn't an in-game way to toggle it on as a result but I'd be happy to add some form of locale selection once things are more mature (it can be turned on for the GUI at that stage)

Partial translations are fine. As you can see the PL one is just a proof of concept!

Names only need to be translated for display - they don't need to be stored that way. If the lookups prove to be a performance issue they can be cached reasonably easily on a per-script basis.

Namespacing things isn't super likely - it's do-able but just adds more indirection and I'm not sure it really helps with the verb/noun side of things.

As you mention at the bottom could be able to be differentiated by variable substitution. I'd rather keep the strings {KILLER} named rather than using {0} and co for easier 3rd party translation and just use basic replace script-side. Getting that variable substitution in is pretty important because many languages don't have a compatible way of saying things to English.

[asumagic]

I've done a translation helper class to provide the variable substitution as well, and it performs reasonably fast: Approx ~3x slower (without the parsing stage) than a barebones solution in multiple of my tests (long lorem ipsum and short text). Since not everything needs to use this it looks fine to me.
The parser shouldn't be very slow. It doesn't care about the untranslated, unformatted version of the string and is pretty basic.
It is pretty straightforward and it allows variables to be automatically translated, e.g. {KILLER} has killed {VICTIM} with {@ITEM} - ITEM will be automatically translated because of the @ character. Of course, reordering variables works too.
The FormattedTranslation should ideally be built once, and the usage looks like this:

string[] order = {"KILLER", "VICTIM", "ITEM"};
FormattedTranslation tr("{KILLER} has killed {VICTIM} with {@ITEM}", order);

string[] replace = {"copper40", "kf9000", "a hammer"};
print(tr.translate(replace));

Unfortunately it doesn't seem possible in the AS version KAG uses to anonymously initialize arrays with an initialization list (according to the wiki, the syntax would be string[] = {"KILLER", "VICTIM", "ITEM"}). That's not really convenient sadly... I assume it would be quite not trivial to update AS?

[1bardesign]

I'm not sure that this translation helper is of huge value - it's an interesting idea but seems like it doesn't cut down much on the code from the simple replace() based version which can pre-translate variables to be inserted as appropriate. If it's not faster and not smaller I don't see much benefit and think that it'd be better to pursue getting the translations used in more places, standardising the syntax for translations and getting a handle on how much text really needs translation.

Updating AS is indeed not a trivial matter, though should be easier than with irrlicht if absolutely necessary because there's at least not thd-side untracked modifications to it as far as i'm aware.

[asumagic]

You're pretty right actually, the replace() method is only about a quarter slower in a scenario that doesn't help it much, and it is actually pretty short to use, so I'll drop my Translations.as file in my branch.

[asumagic]

client_AddToChat doesn't seem to support unicode characters apparently? It's not great for french but this makes it impossible to translate to some languages (it isn't often used, but it is e.g. for resupplies in TTH). Logging also doesn't support unicode correctly according to deynarde. But overall, it looks like GUI functions prints unicode pretty well. It's pretty straightforward for those.

[1bardesign]

Sorry, I missed the more recent comment here.

The "full" unicode support of the engine is unlikely to ever be there; there's stuff like RTL scripts that as far as I can tell will never work with our font engine, decorator stuff and so forth. The unicode implementation inside irrlicht is heavily UTF-16 based (to mimic windows) but as far as I can tell is somewhat buggy. We try to keep things inside AS scripts as UTF-8 and convert to widestrings as late as possible. I'll put both client_AddToChat and the logging functions on the list but I'd be unsurprised if the logging never really supports unicode in the console at least as it's my understanding that the command prompt "wants" system-locale-dependent multi-byte encoding but in the log file we "want" utf8.

[suzumiyasmith]

Need any help for translation? I'd like to do the simplified Chinese part.

[1bardesign]

@suzumiyasmith hello! Any help with translation is welcome but chinese simplified will need some testing of parts of the engine probably and I'm not sure if our font supports it.

• will need what @AsuMagic is working on to be finalised
• will need some engine-side changes to allow the out-of-game GUI to be translated properly
• will need the first set of strings decided on for a dedicated translation work
• will need to ensure the font engine can correctly render chinese glyphs and a suitable open-license font to distribute with the game for chinese users.

If you're willing to be part of this on an on-going basis then getting in touch with Asu directly could be best. Other helpful things would be starting on the testing side of things and ensuring that the substitution system proposed will work for chinese translations.

[suzumiyasmith]

I have noticed today's update, and am really glad to hear about that.

Update log said the translation is available now. However, when I try setting language local to Francais, it seems nothing happened even if I restart the client.

Any idea?

[asumagic]

There isn't a lot of things translated and support is still very early - everything that is hardcoded in the engine (i.e. using irrlicht GUI) doesn't have translations just yet, because I didn't have engine access at the time. The only engine-side change I did for translations is implementing the dropdown menu for selecting languages.

You should see some strings translated with the french locale, for example in the scoreboard menu, some gamemode related strings or help-related strings. Tips should be translated too, but for some reason they seem broken...

[suzumiyasmith]

Thanks. I see. Hard coding may need more works. Thanks for your devotion.

[1bardesign]

The menus will be our next priority for translation stuff I'd say, as it's the other big "important" thing beyond being able to read things in-game.

Tutorials and the like might take a little longer to translate as there's a lot of text there that isn't seen very often, and they might change in the future as they're very rough currently anyway.

[diprog]

Found some problems with unicode:

1. Unicode symbols don't show up in a grid menu description.
2. Unicode symbols make offsets to work incorrectly. Text is not in the center of the screen.

There are no spoilers on GitHub :(

[1bardesign]

Spoilers aren't needed for the most part, but if you'd like to keep comments small you can always create a gist and link it.

Thanks for the reports, I'll see what I can do.

[diprog]

Also generic buttons can't draw unicode too

[diprog]

Any way to translate such global messages? rules.SetGlobalMessage("\nMatch starts in " + ((ticksToStart / 30) + 1));

[1bardesign]

Something like this probably:

s32 seconds = ((ticksToStart / 30) + 1);
rules.SetGlobalMessage(
    "\nMatch starts in {SEC}"
    .replace("{SEC}","" + seconds)
);

[diprog]

Ehm. Server will replace the string and then send it to clients, so clients will translate "\nMatch starts in 30" for example. Yeah, it does not work.

[1bardesign]

@Diprog oh, I see - sorry, I was a bit out of sorts towards the end of last week. Yes this is a big problem with using the global message for anything dynamic I guess.

I think the easiest recourse there would be to convert it to use script-side rendered text and avoid the global message in any dynamic cases - that way we can use the replacement but do it clientside after correct translation.

[diprog]

@1bardesign My stupid way of translating global messages

[NiNuNa]

I could also help, if needed.

[1bardesign]

Just to clarify, which language effort would you be able to help with, @Ni-ice ?

[NiNuNa]

German or Spanish.

[1bardesign]

@Ni-ice - feel free to jump in and help! The RU file is the most comprehensive base json file at current I think. All you need to do is copy that file to {FullLanguageName}_{StandardLanguageInitials}.json (eg Spanish_es.json), remove all the Russian translations and replace them with the translations of your respective language.

It would be prudent to post in the respective language-specific forum and the discord to get help, and then work in a translation-specific fork in your repository. You can collaborate with other users using pull requests between your forked repos (or they can just send you the files, but you're more likely to clobber each other's changes that way) - and when you think the changes are good enough to include in vanilla, you can make a PR here as @Diprog has been doing 👌

If you need further help it'd be best to either write here or ask Diprog and Asu about it in #development on Discord 👍

[1bardesign]

@Diprog I'm not sure how closely you are watching this but the server browser translation stuff is now partially included in the test build, along with terrible google translate strings.

I think I lost some of your existing translations for the browser strings in the process, sorry! 😅 If you have some time this week to check them they are at the bottom of the ru json and could use some attention.

[diprog]

@1bardesign finally, thanks.

[1bardesign]

@AsuMagic @Diprog @Suprejso can we get the existing translations on-par with each other in the next release cycle? RU is furthest ahead but I think there are some remaining "google translate" strings in there!

@Ni-ice are you interested in doing any of those German or Spanish translations or is it too much of a time commitment at the moment? The files are there!

[NiNuNa]

@1bardesign Sorry, I'm totally doing that. I'll start today when I'm finally back home!

[mustafatufan]

I have started for Turkish translation.

[Mazey]

Awesome! If you run into any issues let us know in #development on Discord.

[its-kos]

Hello, looking to update my Greek translation I did some years back as I've noticed its become outdated and some things can be improved. Is there a standard we follow ? Is there a locale file you wish to use as base ? Or do I just copy a random one (say the italian one) and start from there ?

[asumagic]

Right now it's a bit of a mess and I don't know which translation file has the most up-to-date entries. I think the German one should be fairly up-to-date as mugg91 recently improved it.