i18n: Add compatibility for Transifex (industry standard provider for open source translations)

gohugoio / hugo

The world’s fastest framework for building websites.

https://gohugo.io

Apache License 2.0

73.66k stars 7.39k forks source link

i18n: Add compatibility for Transifex (industry standard provider for open source translations) #5341

Open gcushen opened 5 years ago

gcushen commented 5 years ago

Problem

Popular themes for Hugo, such as Academic, offer language packs to enable users to easily change the interface language to their native language.

Currently, the format used for Hugo's YAML language packs is incompatible with leading open source translation providers, such as Transifex.

Thus, Hugo language packs are currently very difficult to manage (Academic has over 18 languages), difficult to keep in sync with the master language file, and without the ability to perform translations via the key open source translation providers.

Proposal

Restructure the YAML language file into a standardized format in order to align it with the standardized form used by other major open source projects (e.g. django-rest-framework).

This would make Hugo and its themes compatible with existing language pack translation management tool such as Transifex, the industry standard provider of language translations for open source projects.

bep commented 5 years ago

"Transifex is a proprietary, web-based translation platform;"

https://en.wikipedia.org/wiki/Transifex

I see very little "open source" in that project/product?

gcushen commented 5 years ago

It's the leading provider of free translations for major open source projects.

bep commented 5 years ago

major open source projects.

Name a few.

gcushen commented 5 years ago

I named one major one, I know many other major OS projects using it, will list them in a bit :)

gcushen commented 5 years ago

Here's a comprehensive list (43 million references to Transifex in GitHub code :D ): https://github.com/search?q=transifex&type=Code

gcushen commented 5 years ago

To summarize, this includes edX, Kodi, and pretty much all the other big open source projects that offer some kind of text interface.

bep commented 5 years ago

OK, thanks.

I have thought about this in the past -- and here is my thought about how we could add support for other formats:

Add adapter that converts (internally in Hugo) to go-i18n's data format.
It should be possible to detect what adapter to use either by filename or by looking at the content
This way different formats can live side-by-side

A nice addition to the above (but harder to implement) is a converter the other way.

gcushen commented 5 years ago

The v2 beta of go-i18n has just been released and supports a new flatter YAML language file format which at first glance appears to offer compatibility with translation providers like Transifex - see https://github.com/nicksnyder/go-i18n/issues/85

Perhaps we could consider just moving to this significantly more popular format which is now supported by go-i18n. My other recent proposal (https://github.com/gohugoio/hugo/issues/5342) would help prevent breaking user's sites.

stale[bot] commented 5 years ago

This issue has been automatically marked as stale because it has not had recent activity. The resources of the Hugo team are limited, and so we are asking for your help. If this is a bug and you can still reproduce this error on the master branch, please reply with all of the information you have about it in order to keep the issue open. If this is a feature request, and you feel that it is still relevant and valuable, please tell us why. This issue will automatically be closed in the near future if no further activity occurs. Thank you for all your contributions.

eighthave commented 5 years ago

Ideally, Hugo would just use one of the plethora of existing, standard, full-featured formats that are supported by lots of translation platforms. For example, Weblate is actually free software and many software packages are translated on it. Here are the formats it supports: https://docs.weblate.org/en/latest/formats.html

Using something like Chrome/WebExtensions JSON seems like a good bet:

eighthave commented 5 years ago

I just looked at go-i18n, it supports JSON/YAML formats that have has some decent support (basically, key/value pairs with one or two levels of nesting):

weblate:
  hello: ""
  orangutan": ""
  try": ""
  thanks": ""

eighthave commented 5 years ago

Mozilla is switching Firefox and many other of its projects to its Fluent 1.0 format, which extends on what gettext and XLIFF offer. They already have parsers in Javascript, Python, and Rust, so writing a Go parser shouldn't be too hard. They are pushing this format for websites also, so it could make sense for Hugo also. I haven't used it myself though.

https://hacks.mozilla.org/2019/04/fluent-1-0-a-localization-system-for-natural-sounding-translations/