Consider other file formats or libraries for strings to support accessibility i18n

[jessegreenberg]

From #985 and #986.

We looked at format.js and fluent.js, which uses file formats with built in support for translation logic. While investigating we found Pontoon - a full stack translation solution. It lists a number of supported file formats and we want to learn more about them. We should also do another broad search for any others.

From Pontoon documentation:

Pontoon is a translation management system used and developed by the Mozilla localization community. It can handle any project that uses one of the supported file formats:

.dtd

.ftl (Fluent)

.inc

.ini

.json (WebExtensions)

.json (key-value)

.po (Gettext)

.properties

.xliff

.xml (Android)

[jessegreenberg]

• .dtd: "Document Type Definition". Typically declare markup, but can include translatable strings. Only basic support for strings with variables. Probably only useful for older legacy Mozilla apps.
• .ftl: Fluent.js - Seems promising, and explored in #986.
• .inc: "Include" file. Often used to organize reusable snippets of code and text. Not typically used for translations.
• .ini: "Initialization" file. Usually used for configuration settings. Supports simple strings and variables. Not typically used for translations.
• .json - "WebExtension and key-value" - What we currently use. WebExtension just follows a specific schema for defining metadata for a web extension.
• .po - "Portable Object" used in the Gettext localization framework. This framework is widely to support translations. Features include comments in the strings file, a standardized way to provide context for a string, and pluralization. But there is no other support for logic. There is a small js libr for it. Here is an example:

  
  msgid "Hello, World!"
  msgstr "Hola, Mundo!"

Context for clarity

msgctxt "greeting" msgid "Welcome, user!" msgstr "¡Bienvenido, usuario!"

Plural form example

msgid "You have %d message." msgid_plural "You have %d messages." msgstr[0] "Tienes %d mensaje." msgstr[1] "Tienes %d mensajes."

- .properties - Usually used in Java apps. Has support for placeholders and comments, but no support for logic or pluralization. Not typical for we apps.
- .xliff - "XML Localization Interchange File Format". Designed to standardize the way translation data is structured. It is commonly used for translation in apps. It has good support for comments and metadata. It does not support logic. One might use xliff to store strings and then use fluent to handle the logic. Less used than JSON. Its benefits are mostly for standardization, cross-platform support, and metadata.
- .xml (android) - Specific to android, not relevant for web applications. The xml file holds strings, but there isn't set up to support logic.

[jessegreenberg]

Next, I searched for some other JavaScript tools for i18n. I found these:

• i18next: A full translation solution. It is widely used. Includes formatting, pluralization, fallbacks, error handling, browser caching, language detection, pluralization, variables, functions (to handle casing), context (a modifier from the sim logic could select a different translation). No built in support for logic, but there are plugins we would use. Like https://github.com/i18next/i18next-icu.
• React Intl - One of the libraries under Format.js, it uses ICU format (#986).
• LinguiJS - Also uses ICU, but boasts a more simple interface. Has tooling for developers to support translation, like command line tools.
• Polyglot.js - A library from airbnb. Not as widely used, and best for smaller, more simple projects.
• Globalize - Uses ICU format (#986). Offers many built in functions for managing dates, currency, and number formats.
• MessageFormat.js - Another library for the ICU format (#986). Also has support for Fluent formats (#986).
• Jed - A JavaScript API for Gettext style formats.
• Rosetta (https://github.com/lukeed/rosetta) - A simple library with support for interpolation and managing strings. No ICU/logic support.

[jessegreenberg]

From what I found in this issue, I think our decision is between Fluent and some library that supports ICU. Here are pros/cons for each:

• Fluent.js
  • Most powerful, expressive because of features like terms and attributes. Backed by Mozilla.
  • Less standard than ICU formats. Uses its own .ftl format instead of JSON.
• i18next (with an ICU plugin)
  • One of the most popular libraries out there. Uses JSON, making it easier to adopt than Fluent.
  • Comes with features we don't need.
• MessageFormat.js (My favorite library for ICU formatting from above options)
  • Lightweight implementation of ICU formatting. Not as coupled to React as others. Managed by the OpenJS foundation.

I don't know if the features from Fluent will be needed by translators and if they are worth using when ICU appears to be a bit more standard.

[jessegreenberg]

@emily-phet @terracoda @jbphet and I just met to discuss the above notes.

We agreed that Fluent is the way to go. It has more features for complex translations than libraries backed by ICU, and we do not see any drawbacks. Fluent is also widely used, so we do not feel a need to use ICU just because it may be more standard.

@emily-phet found a good write up on Fluent vs ICU: https://github.com/projectfluent/fluent/wiki/Fluent-and-ICU-MessageFormat

Granted, the authors are biased towards Fluent, but it describes the differences well.

We are going to port accessibility strings to the Fluent format in a few sims, starting with ohms-law. Then we are going to get feedback from translators to determine how successful this is.