search

openhab / openhab-core

Core framework of openHAB
https://www.openhab.org/
Eclipse Public License 2.0
906 stars 421 forks source link

i18n/l10n #275

Closed martinvw closed 2 years ago

martinvw commented 6 years ago

Continuation of https://github.com/openhab/openhab2-addons/pull/2975#issuecomment-356422898

@cweitkamp sadly due to the fact that this is an endeavor covering multiple repositories we discussed single details here and there. At the last Smart Home Day a few of us discussed localization tools. A short time later I checked existing options and selected crowdin. I did a few tests and after some experimentation we successfully implemented crowdin for projects where the english strings are already available in dedicated key:value files. openhab-android and openhab-core. If my findings are correct the xslt transformation might be our best shot at using crowdin for openhab2-addons. Do you by any chance remember if the script you saw had some enhanced filtering options? The generated english properties files could simply be committed (would conflict with custom key:value pairs) OR they could be directly send to the crowdin api as part of the CI toolchain. The second option is potentially tricky and error-prone and would mean some development and testing work.

@kaikreuzer I'd suggest to move this discussion over into a dedicated issue regarding all openHAB i18n/l10n projects. Is openhab-core a good home for this?

@pgfeller your current code state is technically correct and should be working. The issue at hand is, that all addons should, whenever possible, follow a common coding guideline. That would also cover if or if not @text/ keys are used inside the descriptive xml files. I've rooted for the @text/ approach till I understood the intended localization method. It makes way more sense and I can admit that I was wrong earlier when I told @kaikreuzer otherwise 🙊

martinvw commented 6 years ago

@ThomDietrich for ths xslt file see: https://github.com/openhab/openhab2-addons/pull/3079

ThomDietrich commented 6 years ago

Hey guys, let's try to organize all internationalization and localization efforts involving openHAB repositories here.

Localization platform

As I said in https://github.com/openhab/openhab2-addons/pull/2975#issuecomment-356422898 the service crowdin is the platform we chose for localization. So far I am very happy with the options and functions offered and their support is very friendly and responsive. I'd not want to second-guess that decision, of course we could go into that direction if arguments speak for it.

Localization process

@martinvw asked "should everyone have access to contribute [...] ?"

Translation on the crowdin platform is a two step process. (1) Translators start to translate, rate and discuss translations for single strings, then (2) a user with the "Proofreader" privilege approves a translation. Only approved translations are fed back to the project repository (via PR, this restriction can be disabled but I feel it is an important QA step). With that I think, Yes, everyone should be allowed to add translation suggestions.

In the openhab-android project we've decided to give the proofreader right (for individual languages) to users who were known to provide translations for longer times. Atm I don't think we have to be overly strict in who receives proofreading rights, as long as no random unknown troublemaker is easily able to introduces malicious content.

openHAB projects overview

Project i18n/l10n State
openhab-core / Dashboard Already set up, imho needs some strings rework
openhab-core / HomeBuilder Already set up, actual use of the localizations is still under development by @kubawolanin
openhab-android Up and running!
openhab.ios Status unknown @digitaldan ?
org.openhab.ui.habpanel Up and running!
openhab-openhabian Possible and should be easy to implement
openhab2-addons See below
... List not yet complete ...

openhab2-addons

Check out: https://www.eclipse.org/smarthome/documentation/features/internationalization.html The english base strings are mainly contained in between code (i.e. here inside the xml files) and program logic replaced them when/if a different language is used/available. English properties file are still used in seldom cases (Custom Keys or for strings inside the program code, e.g. error messages)

Problem: Crowdin does (~to my knowledge~ confirmed) only support key-value-pair import.

Solution: The transformation provided by @bjoernbrings generates the english properties files. The next step would be to upload them to Crowdin initially and continuously. As I doubt that committing them to the repository codebase is an option, we need to figure out a way to do so somewhere along the CI/CD toolchain. After generation of the properties files, the crowdin-cli-2 project offers the means to upload them to crowdin. This will update the existing sources and notify translators about new/changed strings to be translated. ~For the localized files crowdin will create a Pull Request as already known and can be merged on a regular basis.~ I'm in discussion with the crowdin support.

Todo: I've contacted crowdin support to see if the described approach is as intended. @kaikreuzer @cweitkamp wdyt? What would be the best way to implement the described steps in existing/easily set up automation?

hakan42 commented 6 years ago

So, whom would I contact to support an additional language? @kaikreuzer ? @ThomDietrich ?

I tried using the "contact" feature of crowdin but could not get it to work...

I'd love to see "Turkish" as an additional language in my openhab installation 😄 :tr:

ThomDietrich commented 6 years ago

The link should open up a chat with my account. Could you try again and if different send me the correct URL? I'll add Turkish for openhab-core now. When the time is right I'll make sure that all projects offer the same set of languages.

hakan42 commented 6 years ago

openhab-core has a few large wall-of-text strings, for example the warning about the "YOUR HOME IS EXPOSED".

Could be break that up into multiple sentence entities so it is easier to translate them and also keep the translations up-to-date when only colors and targets of the html links change?

hakan42 commented 6 years ago

Also, will it be possible that every addon / binding has its own translation subproject at crowdin? I feel that would make assessing the completeness of a specific addon easier, while at the same time increasing the workload on the admins...

ThomDietrich commented 6 years ago

I've expressed the same concerns to others... I'm afraid I can't do more than that. Maybe you want to dig in and help. See: https://github.com/openhab/openhab-core/pull/244#issuecomment-350264244

Regarding Addons: I doubt that would be possible but also think it's better to have only as many crowdin projects as needed. Management of these projects can be time consuming and error-prone (e.g. selecting Turkish in all of them). For translators I think it's also easier to keep track of less projects. The way we are working right now is I think a good direction.

ghys commented 6 years ago

Hi, just chiming in here and let you know I intend to join this effort and provide full i18n/l20n for HABPanel as well, it makes sense since it's an end-user friendly UI. Discussion about the integration of crowdin is currently happening here (but I'm subscribed to this thread too now): https://github.com/openhab/org.openhab.ui.habpanel/pull/256#issuecomment-358060741 Cheers!

ThomDietrich commented 6 years ago

Hey guys, I had a few back and forths with the crowdin support. Sadly there doesn't seem to be a direct option for the openhab2-addons situation:

I did discuss your issue with our devs team but I'm afraid there is no other solution except using CLI for both files upload and translations download. I understand it's not ideal for you, though there is no way to sync translations to git without specifying the source path; we thought of a custom pre/post processor for you but it still will require the integration and complicates the case is that you need certain strings to be translated, not files. :(

So in conclusion we are not able to use the crowdin GitHub integration but have to

  1. Pull the latest commit
  2. Execute the XSLT transformation to get all english source strings in properties files
  3. Execute crowdin-cli2 to upload those files to crowdin
  4. Wait for them to be translated
  5. Download translated files and commit them to the master branch on a regular basis.

@kaikreuzer wdyt?

kaikreuzer commented 6 years ago

Sounds complicated :-( Can steps 1-3 be automated through a Jenkins job?

hakan42 commented 6 years ago

Should be / can most definetely be automated, I did something similiar in another project with transifex. Also the part with downloading only the appropiate translations packages...

If someone needs scripting support, here I am :)

ThomDietrich commented 6 years ago

@kaikreuzer 1-3 should be very easy to automate. Should be a twoliner with 1) Execute xslt, 2) execute crowdin-cli2. Step 5 is what concerns me. We need to retrieve the translations with crowdin-cli2 (easy) and then commit them. Is Jenkins empowered to commit? How often do we want him to create a commit? Do we want to bind the Jenkins job to a fitting condition (like "every sunday") to limit the commit count? Should Jenkins go the extra mile and create a PR rather than a direct commit? How?

@kaikreuzer @martinvw @hakan42 what do you think?

ThomDietrich commented 6 years ago

Everyone, a new crowdin is ready to be translated! https://crowdin.com/project/openhab-habpanel

@kaikreuzer I'd aim for an official announcement after openhab-addons is in the game.

boehan commented 3 years ago

@ThomDietrich for ths xslt file see: https://github.com/openhab/openhab2-addons/pull/3079

Don't know if this is still relevant, but I updated the linked xslt file (also covering channel groups, state/command options). Maybe it's of use for anyone.

Thing Translation Template - DRAFT.zip

cweitkamp commented 3 years ago

We have a public project for OHC at Crowdin: https://crowdin.com/project/openhab-core. I think the new UI is already prepared and partly translated too. One more thing to do is to setup the integration for openHAB add-ons.

cweitkamp commented 3 years ago

The Crowdin project for openHAB Add-ons is ready to use: https://crowdin.com/project/openhab-addons. The synchronization already picked up a part of the bindings. I submitted a draft of the XSLT transformation in https://github.com/openhab/openhab-addons/pull/10636. Currently a manual step to create the property files containing the source strings.

martinvw commented 2 years ago

@cweitkamp do you agree that this issue can be closed?

cweitkamp commented 2 years ago

Yes, perfect. Thanks.