Translation system improvements

[wookiefriseur]

Hey, there 👋

The translation system is working fine but while translating I noticed, that it is probably hard to maintain the code and keep the translations up to date. There are some texts that are not translated and some are translated twice and so on. As it is not a bug or anything it's low priority.

I have some ideas for improvements but I'm not sure yet what would be the best way. Just posting this issue so that we can discuss it or at least have some kind of documentation for later reference. If I have time and it's OK i will try to implement the discussed changes and do a pull request (I need some practice 😉).

Current state

Structure

Pros:

• INI-like structure is good for easy reading and writing
  • allows #Comments which is nice
• all in one external file KF2ServerTool.lc
  • can be replaced and edited manually by the user
  • all translations in one place

Cons:

• all in one file KF2ServerTool.lc
  • the bigger it gets the harder it will be to work with it
  • changes to any language mean a change to the whole file, which is harder to track
  • no easy diff with the default language entries
  • Version numbers cannot account for removed lines and so on, needs easier ways for diff
  • some keys are very long
• the file KF2ServerTool_MissingLocalization.txt is unnecessary debug for the user I think

Code

Pros:

• robust (has default / fallback values)
• INI-like structure is good for easy parsing and generating
  • Structure ( {...} = optional ):
    • Category: ["$LANGNAME{-$LANGDIALECT}", "$LANGCODE"]
    • Entries: "$KEY"="{$VALUE}"
• all in one external file KF2ServerTool.lc
• easy to preview the text output in the code because it is the full text

Cons:

• all in one file KF2ServerTool.lc
  • the bigger it gets the harder it will be to work with it
  • has to be loaded into memory completely, even if the other languages are not needed (not really an issue, size is very small)
• the INI sections are just used to separate the languages instead of using it to structure the file
• keys should be unique, but they are not (multiple duplicates)
• keys should not be changed, but that means that a typo could stay there forever
• the longer a key the easier it is to make a mistake
  • one space too much and it is a different key
  • key has to be typed into the code exactly without any autocomplete
• values contain manual newlines (\n)

Thoughts

Structure and Code

1. move translations into separate folder (for example lang)
2. move the readme header from the KF2ServerTool.lc into a separate readme file
3. move each language into its own file (for example Português_PT-BR.lc)
   • use ISO codes after the language name so there is no conflict (list of some language codes)
   • if someone wants add a new translation, they can copy English_EN-GB.lc and rename their new file
4. keep the INI-syntax, it works well
5. remove \n from the values and do it dynamically so the user doesn't have to
6. change the keys from full English text to variable names
   • Example: before (current state)
     • "This will erase the workshop data base file.\nAre you sure?"="This will erase the workshop data base file.\nAre you sure?"
   • Example: after, structure in INI sections, variable using type and action as UPPER_SNAKE_CASE
     • [MAIN]
     • DLG_CLEAN_WORKSHOP_DATA="This will erase the workshop data base file.\nAre you sure?"
7. extract general expressions from longer texts into separate translations
   • for instance Are you sure? is used ~4 times as part of longer texts and could have its own kv-pair like GENERAL_QUESTIONS_CONFIRM="Are you sure?"
   • do that only for general expressions (sentences that do not need a context)
   • try not to reuse specific sentences or words, because different languages have a different word order
8. replace literal strings in the code with string constants
   • could be part of toolLanguage.pas or have its own constants class
   • if it doesn't find translations, let the program generate the default language file (English_EN-GB.lc) from the constants (including sections, kv-pairs and comments)
   • those constants can be used as the fallback/default translations, if a key is not found or a value is empty
   • makes autocomplete available in Delphi
   • that way, if a key is changed or removed, the program can generate a new English_EN-GB.lc (and a simple diff lets you change the other language files within minutes; could be automated later)

ToDo (last edit: 2019-06-10)

• [x] Maybe place missing localization keys for each language in the primary location file.
• [x] Find text in code that is not being translated and implement the call function.
• [x] Remove duplicate or unused translations.
• [ ] Improve the header explanation of how to translate

[darkdks]

Thank you for your attention, is very good to know that people are wanting to help. :)

You have good points, but I have to disagree about other points. About the keys and all translation in one file: It is easier to load in the application, it is easier to control if any file is missing, it is easy to work with the code, it is easy to see the text in the code. For example if you check the "HERE: Questions, suggestion and issues" topic, you will see that some people simply forgot to put some files that are in the zip. To stop these problems I had to put a check for each file in order to avoid user errors, so controlling a whole folder is more code to worry about. And some people just do not like to put so many files in the server folder. Only 3 files is ok, it's easy to uninstall. About the file size, believe me, it does not make any difference to the computer, we're talking about KB. For the translator I do not think it's harder or bigger, I've worked on translating more than 20 apps/games into my native language, by far the majority of them had a localization system much harder to work than that. In the end it all boils down to how easy it is to work with the location system, how fast to program it is. And remember, the tool so far is maintained by one man, it's a lot of thing to worry about.

If you look for others, the much-used location system is exactly how it works. https://en.wikipedia.org/wiki/Gettext in wordpress for example is the same.

• some keys are too long I agree, but it's easier that way. It's easy to understand what the text is saying, you do not have to search for the meaning of a key. It's easy to implement in code, you just need to write the text by calling the toollanguage and the structure will look for a translation or mark it as required translation. The key is generated automatically (that is why the KF2ServerTool_MissingLocalization.txt file exists) I think the time spent creating keys and const can be used to implement new functions, for example.

Like to implement something in the current way, you just need write some text calling _p() function, and it's all. Missing translations go to translation file after finish the code.

I agree that we have to improve the translation system, but in other things:

• Maybe place missing localization keys for each language in the primary location file.
• Find text in code that is not being translated and implement the call function.
• Remove duplicate or unused translations.
• Improve the header explanation of how to translate
• etc

Also, if you look at community comments, the demand is much higher in novices by tutorials, manual, faq, new functions.

In some other points you are right, but it is a characteristic of the form of localization in general. Each type has its pros and cons, in my assessment this fits best by development time. But the tool is open source, you can modify it, as I said I just thing is not the point of spending too much work now.

I appreciate the attention you're taking with this project!, thanks!

[wookiefriseur]

Oh, didn't expect an answer so fast.

Amount of files

And some people just do not like to put so many files in the server folder. Only 3 files is ok, it's easy to uninstall.

I prefer to put programs into their own folders instead of server files because if you have multiple tools and scripts in the end you will not know what belongs to the server and what is external. I put your tool in a different folder and I am glad that it is possible (except for the steamcmd folder, I will try to write a change so that you can use a custom existing steamcmd folder). I can see where you're coming from when the tool is in the server folder though: Steamcmd, steamapps, and language folder would start to look like too much and nobody would know which file to delete when uninstalling. I won't do any changes here (except for the custom steamcmd location).

Ease of use

It's easy to understand what the text is saying, you do not have to search for the meaning of a key. It's easy to implement in code, you just need to write the text by calling the toollanguage and the structure will look for a translation or mark it as required translation.

Mouse over on constants shows the text and good constant names would let the programmer know what the string is used for. For user made translations they would copy the English or another language file and would see the meaning in the value instead of the key. A comment for the users could be added above each section for more clarity. I'm not saying your system is bad, quite the opposite, it's a cool idea, just thinking about improvements.

It is easier to load in the application, it is easier to control if any file is missing, it is easy to work with the code, it is easy to see the text in the code.

For me it's harder because I'm unfamiliar with the code, so for me there are multiple points of failure (whole text to fuck up 😁) instead of just 1 autocompleted constant name. But that's just what I prefer. In the end it's more important what is easier and faster for you because you are the author and maintainer. So I won't do any changes that would force you to change your workflow.

Final thoughts

As long as there are not that many strings and not that many languages no major changes are needed. As I said, it's low priority.

And even if there are more languages and strings in the future, your proposed changes are a good fix:

Maybe place missing localization keys for each language in the primary location file. Find text in code that is not being translated and implement the call function. Remove duplicate or unused translations. Improve the header explanation of how to translate

I will try to implement some of your points when I have time (I'll put them in the checklist in the first post).

Context

Just to add some context why I came up with these changes at all: 8fff85cfed6e04d5f7e154d86ffbfe22c7eef1ad I fixed some typos (values only) and was thinking about changing the keys in the translation and in the code while it's not too much to change. I also noticed some keys having : or spaces that were missing in the values. I don't know if spaces are trimmed so I added them just to be sure.