Use tryton translation system

[pokoli]

Currently the translation system is a little bit messy:

• Tryton translation system is used for tryton models.
• Babel is used for forms amb template.

IMHO that should be unified into Tryton translation system and drop babel translation system. That will imply that forms and templates strings must be loaded into tryton database.

I've created a tryton patch to simply the addition of custom translation types in tryton [1]. Once this patch merged two new types of translations can be added to tryton: nereid_forms and nereid_templates.

For templating we can use the context to pass the translations to the template and treat it as custom variables. Another option is to create a jinja2 extension[2] that similiar to jinja2.ext.i18n that stores uses tryton database as backend. On both cases Translation set wizard must be overriden in order to search for all the templates in the installed modules and create the not found transaltions. This can be done by searching for _( or {trans} in all the available templates.

For forms, it's difficult for me (as I have a very little knowledge of wtkforms) but maybe we have to develop an extension for wtkforms that integrates it with tryton models and treat fields and error messages like normal tryton fields and error messages.

@sharoonthomas would be great if you can provide some feedback about it.

[1] http://codereview.tryton.org/2431002/ [2] http://www.pocoo.org/~blackbird/jinja2docs/html/extensions.html

Phase I:

• [x] Ability to extract translatable strings from python code
• [x] Ability to extract translatable strings from jinja templates
• [x] ability to extract translation comments from the template for assisting translators
• [x] Ability to extract the builtin messages from wtforms
• [x] Implement updating of translations
• [x] Implement cleaning of translations
• [x] Test cases for extraction from nereid code
• [x] Test cases for extraction from nereid_template
• [x] Test cases for extraction from wtforms
• [x] Test cases for cleaning and updating

Phase II: Fetch translations from tryton instead of using the current system

• [x] For python code
• [ ] For wtforms
• [x] For template
• [x] Tests for python code translation
• [x] Tests for template translation
• [ ] Tests for wtforms translation

Phase III: Check ability to export translations

• [ ] Test export of translations
• [ ] Documentation about exporting translations

Phase IV: Write documentation on how to translate nereid

• [ ] Translating core nereid
• [ ] Translating your own nereid module

Optional Tasks

• [ ] Merging of translation patch http://codereview.tryton.org/2431002/ into tryton. Till then maintain hack in the codebase.

--- Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/1896846-use-tryton-translation-system?utm_campaign=plugin&utm_content=tracker%2F386869&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F386869&utm_medium=issues&utm_source=github).

[sharoonthomas]

I agree with the goal, but I do not have much experience here.

At the moment extraction of translatable strings itself is not a problem as nereid already includes support for babel which does this beautifully. Glad to be of any help in implementing this.

As for the code itself I think we should not make any new idioms, but follow the conventions followed in python applications.

• python code (incl. wtforms): use _('Translatable')
• WTForms builtin messages: We could provide a base class and helper that could translate building message strings. The implementation could use: http://wtforms.simplecodes.com/docs/1.0.1/i18n.html
• Templates: use the i18n extension and plug tryton as the source instead of flat files in babel

As for the syntax my preference would be:

from nereid.contrib.i18n import make_tryton_gettext

_ = make_tryton_gettext('module_name')

class ProductProduct:

    ...
    def some_view(self):
        flash(_('This product is not in stock'))

[pokoli]

See https://github.com/pokoli/nereid/commit/a75fffb337f89a4e74294c39943b0a6ac21d5901

It's not fully functional but have some PoC about extracting translatable strings from templates.

@sharoonthomas do you know some mechanism to get the module (parent folder would be great also) in which a template is defined? (Needed for extracting translations)

Maybe i amb missing template loaders, so I don't kwow if there is some mechanism to get all the available template loaders (maybe in Flask, but not sure in tryton)

[pokoli]

I have updated the branch https://github.com/pokoli/nereid/commits/translations with the extraction of python code translation. I extracted all the current translation without touching any base code (using babel).

The only requirement from translations (of templates and python code) is that they must be marqued with _ function (as any normal python code).

Also sincronizing translations sincronize nereids translations (requires the tryton server to be patched if the review in first comment.

It only misses the extraction of translations from tryton server (that's the harder part i think)

[pokoli]

I can load translations from tryton but is a quite painfull now as I search translations from source string, and I need to filter in which module and file is defined the string (in order to get the correct string). I have no idea how to do this. :(

@sharoonthomas I have one doubt:

• Whats the diference between get_translations from https://github.com/openlabs/nereid/blob/develop/nereid/contrib/locale.py#L10 and get_translations from https://github.com/openlabs/nereid/blob/develop/trytond_nereid/i18n.py#L26 ??? Maybe the first it's only called from nereid application and the other is called from modules? (I have seen you use it in nereid-webshoop.

[sharoonthomas]

@pokoli I think we should now divide the work among us. You could write an API which returns the translated string given the current string and module and I will write the API from nereid to use that translations. Do you want to talk about this ?

[sharoonthomas]

@pokoli and yes you are right about the two files, we need to clean that up too

[sharoonthomas]

@pokoli in my branch https://github.com/sharoonthomas/nereid/commit/7ddb6bbef4e898f57c4cc6ef60e7c45442297eea I have rebased with develop and built further over your patch completing phase I of the tasks. I have added tests and I also cleaned up some of the code.

I have also removed the dependence on the proposed patch to tryton core as we are not sure when it will be part of the core.

To run the tests, you will also need the development version of openlabs/trytond-nereid-test with the commit referenced here https://github.com/openlabs/trytond-nereid-test/commit/e7550b15b263b9159735120903d0e9db41c6fd98.

I will start working on using translations from tryton

[sharoonthomas]

@pokoli I have implemented translation lookup from tryton for both nereid code and templates. It seems to be working too. You can check the translation branch.

[pokoli]

@sharoonthomas not working for me (maybe i am missing something). I have a template with a string marked as translatable and it's correctly imported to the tryton database. I have a website with the catalan locale activated and I doesn't see the translations. I've put some prints in get_translation_4_nereid function but nothing is printing in application.

I have the following in the template:

_('Nereid Tutorial')

I Also tried:

{% trans %} Nereid Tutorial {% trans %}

But i get the same result (no translation)

I have run the test (with the new nereid test module) and they are passing correctly (and i see my prints in the console).

Any idea?

[sharoonthomas]

can you confirm that the application.py you used uses the Babel initialisor ?

from nereid.contrib.locale import Babel
...
...
Babel(app)

Example: https://github.com/openlabs/nereid-webshop/blob/develop/application-example.py#L45

May be we should get rid of this and make i18n part of the core package just as i18n is part of core of tryton

[pokoli]

I missed the initialisor, just added it and I get the same result (no translations).

I will be happier if we get rid of this initializator and make the nereid app work as translatable by default.

[pokoli]

Just to inform that the translation patch is merged to tryton trunk :)

http://hg.tryton.org/trytond/rev/1a94e9f76bca

[sharoonthomas]

:+1:

[pokoli]

I was wondering if we can translate the route urls too.

For example the product url is product/:uri: so in spanish it would be: producto/:uri: and in catalan producte/:uri:/

I don't know if this must be impelemented on the translation system or add an opptional dictionary to the new route decorator introduced on #178 in order to specify the translations of the route.

Thoughts?

[sharoonthomas]

Thought about it, we might have to write a new URLMap. But do you think the django implementation of translated URLs [1] is good ?

[1] https://docs.djangoproject.com/en/dev/topics/i18n/translation/#translating-url-patterns

[pokoli]

So this will imply saving the translations on the database, and import/export via po files and then when loading the routes try to search if the url has a translation. It may work, but i think is to complex.

The other alternative is to use a new decorator which add the translations for the given route. Something like:

@route('/product/<uri>')
@route_translation('es', '/producto/<uri>')
def product(self):

What do you think?

[pokoli]

Just started a PoC on https://github.com/pokoli/nereid/tree/route_translations but i think is not as easy as I thought.

[sharoonthomas]

@pokoli can we discuss translated URLs on a separate issue from using the tryton translation system ? Especially since your proposal indicates that we should not use tryton translation for it ?