Multi-language/I18n support

[proddy]

Support multi-locale for Telnet console commands, MQTT, SysLog messages and the Web. Since there is not enough memory to hold all the translations it will have to be different builds (DE, NL, UK). String literals will be stored in translation files and built at compile time.

@MichaelDvP @bbqkees what you vote for this feature?

[proddy]

@MichaelDvP should we also use the translated names of the devices (Boiler, Thermostat etc...)? Means changing EMSdevice::device_type_2_device_name()

[MichaelDvP]

They are also used for API and mqtt-topics and subscriptions. For web this could be ok, but for mqtt we should stick to en, or add an option to force it. Shortnames/commands are not translated. But i think better leave them as is.

[MichaelDvP]

nice, that gives us plenty of room for the next set of languages (swedish, polish).

I've prepared v3.5.0 for polish translation, do you have someone in mind to translate?

[proddy]

nice, that gives us plenty of room for the next set of languages (swedish, polish).

I've prepared v3.5.0 for polish translation, do you have someone in mind to translate?

@bbqkees ?

[bbqkees]

There was someone who wanted to help but I can't find the right person in my mail anymore.

[bbqkees]

I now asked one of the PL customers if he can help.

[pierafal]

I can help with polish translation if you like. I didn't have time to go through the whole threat yet, but if you could send me some details of what is needed, i'll try

[MichaelDvP]

@pierafal Great, there are two files to edit.

• interface/src/i18n/pl/index.ts for the web, you have to replace the english texts by polish.
• src/locale_translations.h for entities, you have to append the polish translation as , F("translation") to each list before the closing bracket, Scheme: MAKE_PSTR_LIST(name, F(en), F(de), F(nl), F(se), F(pl))

If you want to check the we translations you can use the mock-api, but for the entities you need to compile and flash to emsesp.

[proddy]

@MichaelDvP let's create a section in the 'For Developers' section on the wiki stating the steps needed to add a new translation.

[MichaelDvP]

I've made a PR to the doc with (hopefully) all steps to add a new language. Please check.

[bbqkees]

Which languages do we want to implement in this first round? If we have EN/DE/NL/SE/PL we already covered the most important ones. Others can wait I guess. CZ/HU/IT/FR would be nice but not a lot of activity over there. On the docs you mentioned a Finnish translation but this is not a country I shipped to recently.

[proddy]

It wasn't Finnish (my mistake) but Norwegian! Viking_NO requested it on https://discord.com/channels/816637840644505620/816638754772353095/1004403792529346732

[proddy]

Polish (PL) translation is underway but it'll take a few weeks and will be incomplete. We may decide to comment that language out to avoid confusion when we push 3.5.0 to dev

[MichaelDvP]

Ok, i think remove only the selections. Maybe it's easier to collect the translations in a spreadsheet and generate the headerfile from that. Than we can sort the columns and it's better see missing translations.

(Edit: Sheet removed, please use newer sheet below)

[MichaelDvP]

This is the web translations as table: web.xlsx I'll make a PR with pl and no selections disabled.

[proddy]

that's great if you have a script that can quicky output the locale string data

[MichaelDvP]

It's only export csv and three replace operations in editor, Insert a empty column after tag and export csv to have tag;;shorttext;en,de,nl,se replace ;;by,F(", replace ; by "), F("and replace \r\nby ")\r\nMAKE_PSTR_LIST(

[MichaelDvP]

I've tested the way back, maybe it's easier if we leave line start/end as fields, and replace only multiple semicolon.

BTW: in this spreadsheet missing translation could be seen much easier, @mvjt could you please check lines 353-356 and 407, 408. @pierafal You could add the translation to this sheet if you like, no handling of brackets and quotation marks.

(Edit: sheet removed)

[mvjt]

I've tested the way back, maybe it's easier if we leave line start/end as fields, and replace only multiple semicolon.

BTW: in this spreadsheet missing translation could be seen much easier, @mvjt could you please check lines 353-356 and 407, 408. @pierafal You could add the translation to this sheet if you like, no handling of brackets and quotation marks.

locale_translations.xlsx

Hi @MichaelDvP , the Swedish translations for 353-356 and 407,408 are there. You're missing the NL translations. So the Swedish ones are now in the NL column.

[MichaelDvP]

Thanks for checking, @bbqkees could you please take a look. Updated list: locale_translations.xlsx

[proddy]

Kees is on holiday so I'll see if I can add the NL ones.

[tp1de]

I tried to do my best to adjust some German translations (marked in yellow). I have to admit that I do not understand all parameters and some descriptions are not unique ..... locale_translations.xlsx

[MichaelDvP]

Thanks Thomas, i've added most of them, some WW are not needed since the entities are prefixed with dhw (planned to translate to WW). For a few others i stick to wording from my german Buderus manual (Antpendelzeit, Einschaltdifferenz, etc.) I've also added the web-translations to this xlsx, for NL the later introduced help texts are missing (en-text). I'll make a PR to dev soon with this texts.. locale_translations.xlsx

[pierafal]

hi, i've done pretty much of the web translations but i see some issues. Please see the web.xlsx, a color cells in column F requires some clarification to understand the context. Can someone please review that and comment? The issues:

1. different inflections of words. I.e. In english you have 1 "Second" and for plural you have "Seconds". In Polish you have 1 "Sekunda", 2-4 "Sekundy", and for more than 4 you have i.e. 5 "Sekund". So we have more than 1 inflections for plurals.
2. In english You can have separate words like "System", and "Log" that can work together as a "System Log". In Polish, you can translate separate words respectively "System" and "Dziennik", but when you put these straight together you'll have "System dziennik" and it sounds very weird - inflection needs to be used to have "Dziennik systemowy" which is correct form.
3. i do not see words "official" and "development" to be translated, from the Version Check.

What do you think of creating a screen shots of each WEB page in english to have a context of what we're translating. IMHO a list of words is not enough to create a good translation. We need to understand the context web.xlsx

[MichaelDvP]

Yes it's not easy to understand the context. For web translations it is best to use the mock-api and check the webpage with your translations to see the context.

1. Thiis is pluralization described here, for poland i find Category one/few/many/others here (see definition of d on top) I think for secondes this should be {num} Sekun{{da|dy|d|d}} expanding to 1 Sekunda, few Sekundy, many Secund, other Secund. I don't know what difference is in many/others.
2. Yes, the LOG is only used in combination with System. This is set on many pages for Status and settings: Network Status, Network settings, Mqtt Status, Mqtt Settings, etc. It is a bit redundant because the page headline is System and the tabs are named System Status, System Log. @proddy Any idea how to handle this?
3. Yes, actually only THE_LATEST is translated, official and version is are fixed.

If you don't compile yourself, the v3.5 is now in dev, you can download the bin and test the EN page and check the context. For mqtt you'll find the options here. I think you should not translate clean session, retain flag and response-topic , as they are special terms in mqtt configuration, but translate the always set . The options for single topics gives:

• normal publish is a long json with all parameters: thermostat_data:{"hc1":{"seltemp":21, .....}
• single topic publishes a value for each parameter: thermostat_data/hc1/seltemp:21
• publish to command-topic is: thermostat/hc1/seltemp:21 This is ONLY usefull for a mqtt-broker that is a endpoint and do not republish (like ioBroker-mqtt (if set)). A standard mosquitto will republish this to subscribed ems-esp as command to set seltemp to 21 and this ends up in chaos.

Problem loading is shown on a red snackbar on the web ifsomething goes wrong with loading data. offset, factor are settings for configuring anlaog sensors.

[MichaelDvP]

Tested the pluralisation, it has to be {{zero|one|two|few|many|other}} and added this to xls and web. I have added this so far to the dev build, @pierafal you can test a bin here, but i also made a PR, the missing translation make a fallback to EN.

Updated list with web and entities: locale_translations.xlsx

[MichaelDvP]

I've found that the EN-fallback in web only works only for a missing index-file, not for single word missing. So i set missing texts to english and marked them red in the xlsx. locale_translations.xlsx

[proddy]

I've added the missing NL ones

[pierafal]

Guys, sorry but i'm not sure what should I do to use mock-api. I was expecting doing some tranlslation in a human readible form like xls. I'm still ready to work on that but I'd need more detailed instruction.

[proddy]

Guys, sorry but i'm not sure what should I do to use mock-api. I was expecting doing some tranlslation in a human readible form like xls. I'm still ready to work on that but I'd need more detailed instruction.

did you edit the files in the repo or update the XLS? Sorry, can't remember...

[pierafal]

I edited the xls file

[proddy]

ok, are you able to use check out the source file from the repo, make the edits and push back as a PR? If you've never worked in github before we can always do it manually from the XLS you created but it'll take some time

https://emsesp.github.io/docs/#/Coding?id=adding-a-new-language

[MichaelDvP]

@pierafal Your web translations from xls file are already implemented in the dev. Only a few words are missing, see here. You can flash the dev and check, no need for the mock-api.

Next is to add translations of entities in locale_translations. You can use the xls or edit a local copy of locale_translations.h

[proddy]

the last changes to support Polish were added. Closing this now.