i18n support

[Happy-Ferret]

This isn't a bug report or a feature request. More of a general inquiry or an RFC (in fact, I'd love to debate some implementation details).

I was wondering what your stance on i18n (internationalization. In this instance, a focus on localization specifically) was.

Is this something you'd like to see supported in the foreseeable future?

I currently work on a set of simple i18n libraries for both Go and TypeScript (aptly dubbed Rosetta-Go and Rosetta-TS, respectively).

Both libraries follow the Chrome i18n API (just the basics, so far. No placeholder support, yet).

In its most basic incarnation there's a locales directory. Inside that locales directory are folders named for language shortcodes (en, fr, de, etc. So far, no support for country-specific shortcodes). Inside each of these folders is a messages.json file that adheres to the following basic pattern (where "new_map_title" and "new_map_text" are unique keys, and the contents of the "message" fields the language-specific display text):

{
    "new_map_title": {
        "message": "New map"
    },
    "new_map_text": {
        "message": "This will erase all editor contents. Proceed?"
    }
}

To quickly switch between languages in the browser, the user may provide a lang argument (i. e. www.trizbort.io/app/index.html?lang=fr — or www.trizbort.io/app/index.html?map=hobbit&lang=fr, to load a new map while also switching the language to French).

If a language code is not recognized, the application will launch with the default fallback language (English, for Trizbort) instead. Although here one may also consider checking the browser language (window.navigator.language) against the array of supported languages.

Both i18n libraries will provide a GetMessage(key: string) method to place a translation.

The translation specific code for the screenshots below looks as follows:

actionNewMap() {
    new Window(i18n.getMessage("new_map_title"), i18n.getMessage("new_map_text"), () => {
      // OK
      App.map = new Map();
      Dispatcher.notify(AppEvent.Load, null);
    }, () => {
      // Cancel
    });
  }

The beauty of utilizing Chrome i18n lies in its simplicity, as well as the compatibility with most online translation providers like Transifex or Crowdin.

Screenshots: A first WIP implementation running in my fork.

[henck]

I would be very keen to add localization to Trizbort. The application doesn't have a vast amount of localizable text, so it should be little effort to translate. Later on, it will enable other contributors to add translations over time, without having to touch the actual code.

I am happy to see development of a TypeScript implementation of this, so it'll be easy to drop in. In particular I like "simple" implementations: it is immediately clear how this should be used.

I do find accessibility an important feature. This is why Trizbort has (some) keyboard support, and localization is also part of accessibility.

A couple of questions about rosetta-typescript:

Resource file format

Is there a reason why the message JSON format is this:

    {
        "new_map_title": {
            "message": "New map"
        },
        "new_map_text": {
            "message": "This will erase all editor contents. Proceed?"
        }
    }

rather than this:

    {
        "new_map_title": "New map",
        "new_map_text": "This will erase all editor contents. Proceed?"
    }

I figure it has to do with some standard, but I'm curious to know.

Language switching

I thought it might be nice to be able to switch languages on the fly, rather than opening a new URL. This seems to be rather more of a change to Trizbort than to any translation library; it would have to re-render the GUI with new text, while keeping the user's map open. On the other hand, this is probably a feature that no one would ever use. Once a user picks a language, they're likely to stick with it, I suppose. Perhaps this may be a consideration for your project?

Dependencies

There were a few design goals when I starting building Trizbort:

• Keep it small
• Keep it simple
• Have no dependencies (everything can be done in plain JS, no fashionable libraries, no additional downloads)

Clearly rosetta-typescript could be compiled into Trizbort and require no additional download. On the other hand, the language files will be additional downloads (well, just one). I suppose it will be retrieved through AJAX from the server, after Trizbort first loads. Do I have that right?

[henck]

Also see #30 for a use case where localization would add a lot of value.

[Happy-Ferret]

Ok. I'll try to briefly answer your questions before I endeavor on my daily lunch hike :smile:

Resource file format

I chose the format specifically because it's already being used by Chrome Apps and WebExtensions in general.

Furthermore, as noted before, it's well supported by leading online translation platforms.

Language switching

I thought about that, and it would certainly be a possibility. The main reason I chose to add an additional argument is that it makes working with i18n from my fork's Go back-end quite easy and straightforward.

During window initialization I simply set the Homepage field (what ends up being displayed in the Electron window) like this:

Homepage:       "index.html?lang=" + i18n.GetUILanguage(),

Dependencies

The current implementation utilizes the Fetch API

The following code is what this boils down to (where Translations is an Interface describing the translation file format)

import { Translations } from './types';

export default function load(locale: string): Promise<Translations> {
    const url = `locales/${locale}/messages.json`;

    return fetch(url)
        .then((response) => {
            if (!response.ok) {
                throw new Error(
                    `${response.status}: Could not retrieve file at ${url}`
                );
            }

            return response.json();
        });
}

[Happy-Ferret]

Ok.

For the TS version of the library I've created a slight divergence from the Chrome i18n reference.

I've added the following method

setUILanguage(locale: string): Promise<void>

This accepts any one locale string and, unless rejected (i. e. in the case of an unsupported language), loads the new language and replaces all existing message strings.

That should deal with your Language switching question gracefully.

[Happy-Ferret]

@henck

I just finished the initial implementation this morning (minus the desktop app specific parts).

The source code can be found here (the i18n library lives under src/rosetta. The translation strings under locales).

Any key prepended with app_ is desktop app specific. All the other keys are used throughout both, the web version and the desktop app.

For translation tracking I chose Transifex.

Eventually I plan moving Rosetta into its own distinct git repository and publish it to npm.

Demo

English

https://pedantic-heyrovsky-f84015.netlify.com

German

https://pedantic-heyrovsky-f84015.netlify.com/?lang=de#

One more implementation detail that will probably change over time:

Currently, strings that are untranslated will show up as their keys for that respective language. i. e. if there was no German translation for new_map_title, all a user of the German version would see is "new_map_title". In the future, Rosetta might load the default/source string instead. In that case a user of the German version would see "New map" instead.

[henck]

Great stuff! I had a look at the code just now. It'll be interesting to add this to Trizbort, too.

I am unsure what Transifex does, exactly. Is that where I would go if I wanted to contribute translation strings?

[Happy-Ferret]

Exactly. Transifex is my chosen translation platform (you could also pick Crowdin or any other solution. I just found Transifex the easiest one to set up).

Transifex is where you translate strings and add new languages, whereas all the source strings are defined locally (inside locales/en/messages.json).

This is the translation workflow, in a nutshell:

1. Define source strings in locales/en/messages.json.
2. Use the new language strings inside trizbort.io/Yggdrasil.
   • Either (TypeScript)

     App.i18n.GetMessage(translationKey: string)

   • Or (Handlebars)

     {{{i18n "translationKey"}}}

3. Commit/push changes to master branch.
4. Translate strings inside Transifex.
5. Mark translations as "reviewed".
6. Once a language is marked as "100% reviewed", Transifex automatically sends a pull request to the repository.

[Happy-Ferret]

I also added a short write-up about it to my blog:

https://happy-ferret.github.io/posts/yggdrasil-announcement.html

I've got a busy week ahead of me, but I'm positive I can wrap up Yggdrasil v0.3.0 by the end of the first week of September. Once that is done, I'll work on a patch for trizbort.io adding the same functionality to it.

[henck]

You leave me no choice but to sign up for Transifex to provide you with Dutch and Portuguese translations. A language request should be coming in momentarily.

[Selsynn]

I didn't follow everything but count me on to translate it in French.

[henck]

@Selsynn Translations are for Yggdrasil, a fork of Trizbort, and they're done in Transifex (sign up there -- see messages above). @Happy-ferret will need to accept your French language request. We may be able to move any translations back to Trizbort later.

[Happy-Ferret]

Cool! I'll make sure to add more keys over the coming days. So far, I think I've only added translation keys for the main menu panel and the dialogues.

[Happy-Ferret]

Ah. I see you already figured out how to request new languages 😺 Just added Dutch, French and Portuguese.

[Happy-Ferret]

In the future I'll probably add "description" fields to the source strings, to make it easier to actually locate a language string/make sense of it.

[henck]

@Happy-Ferret You'll still need to accept our joining the various translation groups on Transifex.

[Happy-Ferret]

@henck Strange. I don't see any requests, and I can't invite you by username somehow.

[henck]

There should be some pending requests:

[Happy-Ferret]

Ah! Found it. Was under "Teams" rather than general notifications.

[henck]

Confirmation received.

[henck]

OK, Dutch and Portuguese fully translated. Be sure to add new keys and give me a heads-up.

I have found that there is a Spanish group using Trizbort.io to create JSON maps that they then parse into a format needed for their project, using a custom Python script. They might appreciate a Spanish translation of Trizbort.io/Yggdrasil, as well.

[Happy-Ferret]

I just updated the source strings, @henck and @Selsynn.

There now translations for the Note panel, the Color Picker control, Load/Import error dialogs, and the rest of the desktop app menu (please note that I'm following the Microsoft/Windows standard for those app_* entries. i. e. all words of those app menu entries should be capitalized).

[henck]

I can see the new strings. However, I don't understand which entries should be capitalized. The ones with keys starting with app_ are not capitalized in your own English version.

[henck]

Could it be that You Mean Capitalized rather than CAPITALIZED ?

[Happy-Ferret]

@henck

Exactly. Every single word should start with a capital letter.

i. e. "New Map", "Load Map", "Save As Image", etc.

[henck]

OK, all done.

[Happy-Ferret]

Thanks! I'll review/merge the translations during my lunch break, then (hopefully) add the final source strings by the end of the day.

Shouldn't be that many left, and after that I can finally go back to cherry-picking some patches off of Trizbort.io and work on the i18n patch for Trizbort.io.

[Happy-Ferret]

Pushed another large update.

I think the only ones missing from my version are now the Connector Panel and the Render Settings Panel.

Sorry for the many duplicates. In the future, I'll probably shrink those down to one common key. In the meantime, you can probably just use Transifex' Translation Memory feature to (semi-)automatically translate those duplicates.

[Happy-Ferret]

I've enabled Translation Memory for the entire project now, so all remaining, untranslated strings are unique and indeed without a previous translation.

[henck]

OK, all done for Dutch and Portuguese. I would like to note that the Initial Caps formatting looks a bit strange in Dutch; I don't know if Microsoft enforces it for their Dutch language products, but I've followed your specification.

[Selsynn]

I have some trouble with the difference between Bring to the front/Bring forward, so I will wait to see on the app before trying to translate it

[Happy-Ferret]

Hey, @Selsynn.

You can already test that feature in the web version. It's part of the element's (room, block and note all support it) popup. Simply place a couple rooms, click one of them and choose the option from the menu (see screenshot).

https://pedantic-heyrovsky-f84015.netlify.com/#

@henck Fair point. I thought about that, and it does look strange in most/all Romance languages, too. I'll need to check what Visual Studio Code looks like in Dutch/French.

[Selsynn]

@Happy-Ferret thx I will do the last of the translation tomorrow morning or this evening when i am at home.

[Happy-Ferret]

Added the final source strings. 🏎

[henck]

... and done.

[Happy-Ferret]

Awesome!

FYI. I already started prototyping code for v0.4.0 of Yggdrasil, which will include Chrome OS support. Surprisingly, there's not a lot of necessary changes. Just a couple CSP errors (I'll fix these by vendoring the external JS libraries in index.html).

Interestingly, Rosetta-TS works out of the box. It's using the navigator.language fallback to pick the correct language. I could either leave it like that or, better yet, extend Rosetta-TS to return chrome.i18n.getMessage() whenever that method is called under Chrome OS (while also nulling the JSON loader).

[Happy-Ferret]

@Selsynn

Do you need any help with the remaining translations?

[Selsynn]

@Happy-Ferret Just needed a reminder that i didn't finish it. Will do it right now. It's done now.