RTL and lang attributes in the localization system

[bjubes]

Currently in order for a language to support RTL (hebrew and arabic), they must have rtl=true as their first line in the lang file. The lang= must come next if a language has a "localized localization", meaning that English (US) comes up in the language dropdown instead of en_US.

In #1475 there was discussion of redesigning the system so that the rtl and lang attributes could appear anywhere in a file. I think this is a much better design considering order doesn't matter in the rest of the file, so it makes sense for the entire file to act as an unordered dictionary.

However, this new approach has a flaw. the localization system will have to read the entirity of every file on start, as opposed to just english and the one selected in settings. Currently we have over 50 languages, so the increase of processing time will be immense.

so here are the possible solutions off the top of my head. Please comment on which one you think is best.

• Keep the system as is, just make it arbirary that the lang and rtl attriubtes must appear first. I will fix up the code in localization table to do this more cleanly, right now its kinda a mess.
• Scan every file on start, but on a seperate thread. This will be similar to how world is loaded, using a Thread object and couroutine which yeilds to it. I don't know how long scanning through 50 files which contain potentially hundreds of strings, but I can make it prioritize english (default) and the langauge chosen in settings first, and the rest will load in the background as the game runs.
• Keep rtl as the first attribute if required and move "localization localization" to a separate file (say lang_codes.lang?) where it maps every code to its name e.g. en_US=English (US)

[NogginBops]

I would move language info to another file that has the localized localization and the path to the lang file and so on. Then you just need to read one file to know all basic info about all languages.

[Ohda]

A conf file so we don't need to check all the languages files to initialize the scene.

[bjubes]

I think we can just assume all lang files exist inside the localization folder and are the name of their code as we have now. But I like the idea of a full config file seperate from any lang files maybe something like this?

<languages>
    <language code="en_US" name="English (US)" rtl="false"/>
    <language code="en_UK" name="English (UK)" rtl="false"/>
</languages>

[bjubes]

I went ahead and created a PR using this technique, feel free to suggest changes on it