Implement language translation and investigate improved tools

[isedwards]

Site administrator resposible to translation of overall site.

Country office responsible for local translation of content.

Investigate user interface tools for working with translations, e.g. wagtail-localize.org

[istride]

wagtail-localize

Seems quite immature because not comprehensively documented. Does a fairly good job of extracting strings that need to be translated, but there is an issue regarding ListBlocks. Fields that do no need translation were also extracted; could not find a way to ignore certain fields (missing documentation). Allows page content and templates to be translated. Allows PO files to be downloaded/uploaded. Allows translation to occur via Wagtail Admin. Git integration with Pontoon provided by wagtail-localize-git.

wagtail-localize-git

Outputs PO files into a git repository to integrate with Pontoon. Hardcoded to use the master branch Probably could be made to work with Crowdin if there were more configuration options around the expected structure and content of the PO files in git. Welcomes pull requests to support other integrations, but without a documented path for doing so. Seems less mature than wagtail-localize.

Crowdin

Requires access to an entire Github account (all repos public & private). Repository selection occurs on the Crowdin side and you must trust that they will not do anything with the rest of your account! Requires 'source' files to be uploaded i.e. the strings in the 'base' language e.g. English. Crowdin expects that it will create the files with translated content beside the base language files. This is significantly different from Pontoon, which expects the translated files to be created already (with missing translations) and then to fill in the translated content.

Summary

Managed to get an end-to-end solution implemented with Pontoon, which worked as expected. May not be too much work involved to make wagtail-localize-git generic enough to be used with Crowdin. Likely to need a solution for the ListBlock issue. Essential to find out whether fields can be ignored and how.

[istride]

Implementation branch: wip/wagtail-localize-test Shared git repo for sync with Pontoon: ianstride-iogt/wagtail-localize-test

[tomdyson]

Hey @istride and @isedwards - you're right that wagtail-localize is new and needs better documentation, but it's in production on some large sites. We'd love to work with you to make it better. Ping me (@tom) on the Wagtail Slack - https://wagtail.io/slack - if I can help.

[isedwards]

Hi @tomdyson - thank you for getting in touch, great to hear from you.

We're currently using just wagtail-localize. We have an issue when using wagtail-localize with Wagtail multi-site: https://github.com/wagtail/wagtail-localize/discussions/380

As a temporary workaround I'm considering overriding the wagtail-localize template for /admin/locales/<id>/ to hide the option to "sync from" (otherwise, it's likely that our admins will choose this option and we'll end up with hundreds of extra versions of the sites that are not being used).

@istride - we should revisit the additions you were testing in wip/wagtail-localize-test in the next couple of months to see if we can improve the workflow. If we do use wagtail-localize-git then we can update the branch to use a UNICEF or IDEMS repository for the translation files.

[tomdyson]

Ah, thanks - I'd missed your discussion with @kaedroho in the localize repo.

[kaedroho]

As a temporary workaround I'm considering overriding the wagtail-localize template for /admin/locales// to hide the option to "sync from" (otherwise, it's likely that our admins will choose this option and we'll end up with hundreds of extra versions of the sites that are not being used).

One quick and easy way to do this would be to swap wagtail_localize.locales with wagtail.locales in INSTALLED_APPS. These apps just provide a UI for the wagtailcore.Locale model, the only difference is the "Sync from" field.

My plan is to move the sync from feature into Wagtail itself (and remove wagtail_localize.locales). I'd like to find solution to this problem before we do that though (and also https://github.com/wagtail/wagtail-localize/issues/313)

[dannyparsons]

I've investigated the link to CrowdIn with @istride. It can connect to a GitHub repo in the same way as Pontoon therefore it should also work with wagtail-localize-git.

I forked the test GitHub repo that @istride set up containing the translation files from Wagtail and connected it to my CrowdIn account. CrowdIn correctly read the strings to translate and detected the existing French translations. CrowdIn pushes translations back to the repo via PRs on a separate branch. Here is the PR from CrowdIn when I changed one translation https://github.com/IDEMSInternational/wagtail-localize-test/pull/3/files

There is an issue that CrowdIn has changed the header on all of the .po files. I assume the added lines e.g. X-Crowdin- will not be an issue but it has removed X-WagtailLocalize-TranslationID which is probably going to be an issue for reading back into Wagtail?

I have contacted CrowdIn support to ask about this and if there's a way to prevent it from editing, or at least removing, items from the header when updating the .po files. Will report back the response.

[dannyparsons]

Also, is there a preference between Pontoon and CrowdIn? CrowdIn has good crowd sourced translations and allows any user to suggestion translations to open projects. It is free for not-for-profit organisations. I believe Pontoon is open source and requires hosting yourself. This means any users will need to sign up specifically for your installation of Pontoon in order to contribute to translations. I don't know if this is an advantage or disadvantage for IoGT. I have personally used CrowdIn for a couple of projects and found it very flexible and strong on translation management. I don't have any experience with Pontoon.

[istride]

A method for translating that involves spreadsheets may be preferred, initially, as it is closer to the existing process. Anything we can do to automate this kind f process would be very helpful. In that case, poxls, as identified by @dannyparsons , may be useful.

[geoo89]

I implemented a prototype for importing/exporting translations via spreadsheets: https://github.com/geoo89/iogt/tree/feat/translation_xlsx

@ehmadzubair is going to make this prototype production-ready.

[sheralim012]

@ChrisMarsh82 I think @geoo89 can comment better about the status of this ticket.

[geoo89]

We have working processes for all localization on the site.

• wagtail-localize for content (has some limitations wrt Stream fields, e.g. answer choices for questionnaires)
• PO files for site strings that are part of the repositories
• translation-manager in the CMS for site-strings that have recently be added and don't have an official translation, for the country offices to translate.

There are definitely opportunities for improvement, but moving this to the backlog for now.