WebUI translation (localization) i18n l10n

larjona commented 11 years ago

Hello I could put workforce in translating the Pump.io web interface into Spanish. I am not sure if translating the files in /public/template folder would be enough (are there other files with strings presented to the user?, and how they would be submitted here:

just the same as code? (clone, make branch, create public/template/es_ES folder and put the translated files there, and merge request?)
Using a web localization platform as Translatewiki or Transifex? I'm afraid I have not enough skills to step ahead for i18n coordination, but I could also join such a team if people with more experience lead it. Regards

evanp commented 11 years ago

This is great! I haven't set up translation mechanisms in the code yet. translatewiki.net does the translation for StatusNet and would probably be a great place to get help for pump.io.

One thing I don't know is if there's an easy translation library for node.js. Anyone seen one?

tsyesika commented 11 years ago

I think this is tightly related (if not a duplicate of) https://github.com/e14n/pump.io/issues/415

larjona commented 11 years ago

Not duplicate, IMHO. There are two questions in #415 and this issue is for the 2nd question (translation of the WebUI).

evanp commented 11 years ago

I think this is for i18n on the Web UI and other strings in the system; I'd like to leave it separate.

ghost commented 11 years ago

I found node-gettext and it seems there is other gettext implementation. node-gettext seems to lack a way to extract all the messages from sources, like xgettext does.

There’s also https://npmjs.org/package/messageformat that seems to use ICU formatting.

larjona commented 11 years ago

I'm afraid I cannot help too much on the developers' side, but I found this: https://github.com/mashpie/i18n-node

OTOH, there are already "translatable" files in /public/template/*.utml. I'm not sure if some node.js i18n is needed to take advantage of localized templates or we just need similar templates stored in different folders (one for each language). If this is the case, Evan, could you ask for a project in Translatewiki? (not sure if Translatewiki handles UTML files though). If you prefer other person to deal with Translatewiki people, tell here and I can write to them but I'm not sure I can answer technical details if they ask. Regards

blankoworld commented 11 years ago

I'm also interested by translating application (to my side, in French).

I began manually for my users. It's hard and any git update will lost translation…

Is there any advancement about this ticket state?

argovaerts commented 11 years ago

I am interested in a Dutch translation. Are there any plans made?

larjona commented 11 years ago

Maybe we can setup a translation environment and begin translations even if there is no i18n yet. My proposal is:

Leave the /public/template/*.utml files there so they will be the files used in the webUI.
Then, create a /public/template/en_US folder with a copy of those *.utml files. Those files are the "source" files for translations. A script can be set to update the files from parent folder every night or whenever.
People wanting to start a new translation should open an issue here (we need a label, for example "translations" or "l10n" ). A folder for a new language would be created using the language and/or country code, for example, /public/tremplate/es_ES for Spanish of Spain, es_AR ofr Spanish of Argentina, tr for Turkish).
We setup the l10n environment in Transifex (right now I'm testing this) or if other person has a better proposal and can handle it, for me no problem. So translators would have updated source English files from /public/template/en_US folder via Transifex, and their work would be stored in /public/template/xx_YY folders via Transifex.
Discussions and coordination on existing translations would happen in Transifex, we open issues only for creating/removing languages and for i18n issue. If we do like this, until the moment that any i18n guru comes to help here and convert the webUI in a multiingual site, we could provide translated templates to people wanting to setup a pump.io server with the webUI in other language different than English (overwriting the /public/template/*.utml files with the contents of the folder of the desired locale. Note that the original English files will be at en_US so nothing is lost). I'm sure that soon or later it will be possible to integrate translations in the logic of pump.io so every site can be multilingual, but in the meantime, translators can begin to work, and people wanting a pump.io site in a diferent locale know what to do. I could setup the Transifex project and do the coordination tasks (it would be my 1st try on it, only translator until now, but if nobody else steps ahead, I think I can do it), write instructions for translators and for people wanting to change the language of their pump.io server. What do you think?

tsyesika commented 11 years ago

I'm a bit unsure if transifex is a good idea, I contacted them about year ago and due to the language (na'vi) not having an ISO code yet, they refused to add it. Using transifex will make it difficult for translators like me, I might contact them again and see if there is anyway to resolve this and also I will look into other translating platforms.

larjona commented 11 years ago

For what I read in the docs, maybe we can add the language ourselves; I'll try to do some tests tonight and will tell here my results. Translatewiki is othes platform that I know as translator, but you need to contact them in order to create the project and I felt shy (too early without knowing Evan's opinion about all this?). But as I said, I welcome other proposals.

tsyesika commented 11 years ago

I've used webtranslateit to translate diaspora and pootle, I know webtranslateit isn't FOSS though and pootle I believe you have to deal with setting up your own instance. I'll keep an eye on this though, if we can add our own language to transifex then awesome I'm all for it!

larjona commented 11 years ago

I would prefer not to use non free/open source software, nor having to setup a server. I was wrong with Transifex: it's not possible for us to add any language. We should ask the Transifex team (or open an issue), giving them all the info as explained here: http://help.transifex.com/intro/faq.html#where-does-the-language-list-come-from-how-can-i-add-my-own I have looked in TranslateWiki and it does not include na'vi yet, so if we want to use Translatewiki, we have to ask for creating the project, and also adding the language (probably providing similar information). The same happens for the Pootle server in http://pootle.locamotion.org and for https://translations.launchpad.net OTOH, many projects use different plaforms for different languages; we could write in the README.fortranslators.md the languages that are translated in the platform that we choose, and the languages that use a different platform or method (explaining it). @xray7224 : can you handle the situation of na'vi (I mean, to gather all the information needed and ask Transifex to add the language again, or ask to any other free software platform to do the same?. Other people: any thoughts?

tsyesika commented 11 years ago

Sure, there could be a way round it by using art_nvi (art = artificial) I believe that's what was done on webtranslateit, art is a valid ISO code.

larjona commented 11 years ago

@evanp suggested to use TranslateWiki (as it was used for StatusNet), so I have opened a support request in Translatewiki.net to add pump.io as project (http://translatewiki.net/wiki/Support#Request_to_add_project:_Pump.io_34587 ). We will see if they accept and what we need to do.

nemobis commented 11 years ago

Worth asking: is there some possibility you could switch to another localisation format, one supported by translatewiki.net? That would make the process quickier. StatusNet used gettext, there are also saner ones. :) https://www.mediawiki.org/wiki/Help:Extension:Translate/File_format_support

To save some time Evan could also give push access to @siebrand and @nikerabbit as soon as possible.

larjona commented 11 years ago

Not sure if translate-toolkit's html2po // po2html may do the trick (just tested a bit and it seems yes) or @evanp would prefer to do it in another way (maybe not only for the utml templates but including i18n suppport for all the messages presented to the user around the interface, if there are other).

tsyesika commented 11 years ago

Just to note I've spotted this node.js library I don't know if it's any good

https://github.com/airbnb/polyglot.js

I noticed it due to Ghost (blogging platform) using it. It seems to support the basics inc. plural translation. Might be worth looking into?

larjona commented 11 years ago

Dianara, a pump.io desktop client, includes translations to several languages: http://gitorious.org/dianara/dianara-dev/trees/master/translations There are other clients (Pumpa and Impeller) but I'm not sure if they do so. I just post this here to be taken into account for future WebUI translations (if we all use the same terminology that would be great IMHO).

adslibre commented 11 years ago

I'm interested in a Brazilian Portuguese translation. Anyone?

nemobis commented 11 years ago

We're still waiting for some localisation format to be adopted, AFAICS.

larjona commented 11 years ago

The friends from Translatewiki closed the request for adding pump.io until we have the internationalization done.

OTOH, I've read these documents from Mozilla about localizing node.js web services:

https://hacks.mozilla.org/2013/04/localize-your-node-js-service-part-1-of-3-a-node-js-holiday-season-part-9/ https://hacks.mozilla.org/2013/04/localization-community-tools-process-part-2-of-3-a-node-js-holiday-season-part-10/ https://hacks.mozilla.org/2013/04/localization-in-action-part-3-of-3-a-node-js-holiday-season-part-11/

And I think we could use their approach and tools, they maintain the i18n modules and tools and the l10n part is gettext and PO files which are widely known and supported. I have setup a local pump.io instance for test purposes, and I'll try to follow the guide and implement i18n, but my dev skills are not very good so don't expect quick results from my side...

larjona commented 11 years ago

Hi all I've tried the Mozilla way for internationalizing node.js apps, and I think it could work for pump.io. I followed the guide (the links posted above), and only found 2 problems: 1.- The Mozilla people have created a script to extract the gettext strings into a .pot file (extract-pot) but it doesn't work in our case, because that scripts looks for .ejs files (and most of our strings are in .utml files). So I used the traditional gettext tools to extract the strings. In Mozilla instructions, instead of the line

./node_modules/.bin/extract-pot --locale locale .

You can write:

find ./public/template -iname '*.utml' | xargs xgettext --language=PHP --from-code=utf-8 -c --keyword=gettext --keyword=ngettext:1,2 --keyword=pgettext:1c,2 --keyword=npgettext:1c,2,3 -o ./locale/templates/LC_MESSAGES/messages.pot

2.- At the beginning and in the last step, the instructions recommend to put this code in "our application":

var i18n = require('i18n-abide');
app.use(i18n.abide({
  supported_languages: ['en-US', 'de', 'es', 'zh-TW'],
  default_lang: 'en-US',
  translation_directory: 'static/i18n'
}));

The problem is that I don't know in which file should we put those lines. Anybody can help me?

larjona commented 11 years ago

Got it! It's in /lib/apps.js L61

var i18n = require('i18n-abide');

L220

        app.use(i18n.abide({
            supported_languages: ['en-US', 'de', 'es', 'zh-TW'],
            default_lang: 'en-US',
            translation_directory: 'static/i18n'}));

Result: l10n_es

But it's too late to prepare a pull request! I'll try to do it tomorrow.

larjona commented 11 years ago

I just subbmitted a pull request #863 It works only for layout.utml, not sure why. If I add gettext commands in, for example, main.utml, I can extract the strings, translate them, generate the JSON, etc, but when I display the page, I get an error "Undefined command gettext". I have added a /locale/README.txt file explaining the process to add a new language, extract the strings, and so. I hope somebody can take this and make it work for any template, not only layout.utml (probably I put the javascript code in wrong place, but cannot figure where to insert it).

larjona commented 10 years ago

Related: #893

holisticode commented 10 years ago

I am a bit confused here about what's the status here.

Is pump.io ready for internationalization?
Does a Spanish translation already exist?

If we get a green for our project, I might be of help translating to Spanish

larjona commented 10 years ago

@fablife, pump.io is not ready for internationalization yet. There are some proposals (pull requests) in this issue, but AFAIK the pump.io code is being migrated to CoffeeScript, so probably the i18n patches have to be rewritten. This issue is marked for 0.4.0, I hope @evanp integrates i18n soon.

About the Spanish translation, the pump.io instance https://mipump.es is translated to Spanish, but I think that the maintainer translated the html templates directly to Spanish, so his code is not internationalized either (only in Spanish). You can contact him via email soporte@mipump.es or pump.io (https://mipump.es/luisgf).

If you are interested in contributing Spanish translations when pump.io is ready, just track this issue, I'll shout when we can start :)

larjona commented 8 years ago

With all the changes of the code that have happened in the last months, and the migration from UTML to Jade, I guess this issue has to be worked from scratch or something is recuperable from #863 I've found this: https://github.com/thomasklemm/jade-i18n I'm not sure I can put time on this issue, leaving the info here for future contributions. Any help is very appreciated.

clacke commented 7 years ago

Duplicated in: https://framagit.org/compa/compa/issues/2

ghost commented 7 years ago

The i18n translation should be occur in the Frontend and only in some parts of Backend (like email notifications), is a good idea the package for i18n use a JSON format because can be used by back and front

ghost commented 7 years ago

I think i18next is a good option, because support nodejs and browser, has many plugins/backends and play good with jade(pug), is compatible with: https://pootle.locamotion.org and others like https://webtranslateit.com, has utils for convert from/to gettext.

https://www.i18next.com https://www.i18next.com/plugins-and-utils.html

nemobis commented 7 years ago

Gettext is also supported by translatewiki.net. https://translatewiki.net/wiki/Translating:New_project Comparison: https://docs.google.com/spreadsheets/d/1t6Zo9OmlRCH3N9dOumnWn0w8QrgI9Is2zQXSJB7K-QI/edit

ghost commented 7 years ago

@nemobis support .JSON? because the primary format for i18next is JSON

Nikerabbit commented 7 years ago

Translatewiki.net supports JSON, but not in the i18next format if you use any of the special features such as plural or context support. MediaWiki has a much better inline support for those features which doesn't create an unpredictable set of keys that differ between languages.

ghost commented 7 years ago

@larjona what structure you think is better?

- locales/
- - es.json
- - es-MX.json
- - pt.json
.... etc

or with namespaces

- locales/
- - es/
- - - common.json 
- - - email.json
- - - activity.json
- - - account.json
- - es-MX/
- - - common.json
- - - email.json
- - - activity.json

larjona commented 7 years ago

El 3 de noviembre de 2017 17:25:50 CET, Camilo QS notifications@github.com escribió:

@larjona what structure you think is better?

- locales/
- - es.json
- - es-MX.json
- - pt.json
.... etc

or with namespaces

- locales/
- - es/
- - - common.json 
- - - email.json
- - - activity.json
- - - account.json
- - es-MX/
- - - common.json
- - - email.json
- - - activity.json

second one, with namespaces, looks better organized and simpler IMO. Laura Arjona Reina https://wiki.debian.org/LauraArjona

ghost commented 7 years ago

@larjona but for translated many languages could be add complexity (many files), and it could be repeated, example: activity:{{actor}} reply a note from {{author}} and email:{{actor}} reply a note from {{author}} vs without NS: {{actor}} reply a note from {{author}}

larjona commented 7 years ago

El 3 de noviembre de 2017 20:59:11 CET, Camilo QS notifications@github.com escribió:

@larjona but for translated many languages could be add complexity (many files), and it could be repeated, example: activity:{{actor}} reply a note from {{author}} and email:{{actor}} reply a note from {{author}} vs without NS: {{actor}} reply a note from {{author}}

Ah, I thought that namespaces were mandatory and the difference was between putting all languages in the same folder, or one folder per language. Sorry for the confusion.

Can we make only one file with all the strings, in per-language folders? (Similar as in Android apps: there is values/strings.xml and then values-es/strings.xml, values-fr/strings.xml etc)

In any case, I always prefer the easier way for the devs/integrators. Translators can adapt themselves to whatever the setu is and generally are happy to just "be able to translate".

Cheers Laura Arjona Reina https://wiki.debian.org/LauraArjona

ghost commented 7 years ago

@larjona Yes, it's better

Nikerabbit commented 7 years ago

In any case, I always prefer the easier way for the devs/integrators. Translators can adapt themselves to whatever the setu is and generally are happy to just "be able to translate".

Not true really. Translators are happy when they don't need to concern about these details at all. That's why the online translation platforms are popular and successful.

Having multiple files is better for larger projects, as it is easier for translators to choose what to translate first and perhaps not spend efforts on some rarely used and/or hard to translate messages.

Some repetition in strings isn't bad. Translation memories solve exactly that problem.

ghost commented 7 years ago

The i18n support is almost done, if anyone want test it, please see the branch feature-i18n on my fork.

This support client and server side, but en server side only translate email notifications (I thinks it is not necessary for anything else) and in frontend the templates is in progress, initially the language is set from request header, but the user can change in 'account' page, this is also useful for third-party apps, They can be set the app language according of user preferences from user->language in API response.

Some image:

i18n

pump-io / pump.io

WebUI translation (localization) i18n l10n #502