srl295 commented 8 years ago

Kibana Globalization

Note: These requirements were originally written up over a year ago. Take any specific implementation details with a grain of salt. Phases that were already implemented should be reevaluated to make sure they still make sense, especially in light of the new kibana platform work.

Purpose

Provide translation readiness in Kibana for view text (user data is a separate issue)
A core translation layer that is framework agnostic and consumable in all parts of Kibana
Provide plug-and-play capability (i.e. code free) for localization engineers when providing translations of different languages
Delivered in a phased approach

Architecture Diagram

This was our original plan, but it might need to change dramatically for the new platform

Design

Phase 1

Note this phase was completed, but it's possible the implementation might need to change for the new platform. Most of the code lives here

Purpose

Implement the internationalization engine (I18n class) which provides a means to supply translations in a standard format that it not dependent on a localization framework
i18n engine manages all locale translations; registering all translations and loading the correct locale when required
Translate the Kibana welcome message which proves that the I18n class provides translations as registered for Kibana

Decide Language for Translation Algorithm

The algorithm which decides the language for loading translations is as follows:
1. First, do a direct comparison with the highest priority locale in the HTTP header “accept-language” against the registered translation languages. If comparison found then this is the language returned.
2. Next, do a partial comparison of this locale whereby you try and get a language code comparison against the registered translation languages. For example, "fr" or "fr-FR" is used for all regions of French if comparison applies. If comparison found then this is the language returned.
3. Next, repeat step 2 and if need be step 3 for the next highest priority locale in the “accept-language” list.
4. Continue, until match found in step 2/step 3 or end of the “accept-language” list. If no match found then return empty language.

I18n Class

Manages the language translations for Kibana
Responsible for loading translated content per language
The translations file are JSON files with a flat keyspace, where the keys are unique. This uniqueness between translation plugins could be achieved by prefixing the keys with the plugin name. The key signifies the translation ID which would be referenced in translatable files (like JS, HTML etc.).
The key value is the translation string
Example translation JSON file

en.json

    {
        "UI-WELCOME_MESSAGE": "Loading Kibana",
        "UI-WELCOME_ERROR": "Kibana did not load properly. Check the server output for more information."
    }

Core Kibana plugins like ‘kibana’ and ‘status_page’ could come with their own English translations bundled in
API:
- Return all translations for registered locales
  - getAllTranslations()
  - A Promise object where keys are the locale and values are Objects of translation keys and translations
- Return all translations registered for the default locale:
  - getTranslationsForDefaultLocale()
  - A Promise for an object where keys are translation keys and values are translations
- Return translations for a suitable locale from a user side locale list of BCP 47 language tags:
  - getTranslations(...languageTags)
  - A Promise for an object where keys are translation keys and values are translations
  - This object will contain all registered translations for the highest priority locale which is registered with the i18n module
  - This object can be empty if no locale in the language tags can be matched against the registered locales
- Register translations:
  - registerTranslations(\<absolute_path_to_translation_file>)
  - The path to the translation file is registered with i18n class

UiI18n Class

Handles the interaction between the UI/server and localization (I18n class)
Maps the accept-language header to BCP 47 tags
Fetches the language as requested where the locale is decided by the translation algorithm
It also substitutes any missing translations with the default locale translation

Tool for Verifying All Translations have Translatable Strings

Grunt run task tool that tests all translatable strings have a translation (i.e. all translation IDs have a corresponding translation string)
This could be run by CI to verify globalization end-to-end capability
A possible solution:
- For non-angular constructs: Enforce a pattern to be used. For example a i18n(<key>) function in the Jade template. A tool can then be used to find such pattern and extract the keys to file
- The keys in the key file(s) would then be checked against the language translation files registered

Deliverable

Translate the start-up message (“Kibana is loading ...”) and error message in the Jade template (Kibana Jade template)
Translation message will be loaded using the language as per the deciding algorithm. For phase 1 the default 'en' langauge will be used.
I18n unit tests
Tool can be run by CI to verify translation IDs have a corresponding translation string
Kibana core plugin registers its translation file during the initialization phase. The translation file contains strings for the welcome message and the start-up error message.

Phase 2

Purpose

Integrate a framework like i18next or formatJS with the i18n engine, or build our own. i18n framework needs to be independent from any UI framework (angular, react, etc).
Provide example for enabling translations in an AngularJS view and a React view by creating a POC for each
Implement a tool to generate the translation plugin
Documentation

Translations Available Client Side

Make the translations available on the client side by:
- Embedding the translations in the initial HTML payload:
  - Reduces complexity and removes a round trip
  - Evaluation required to test if it performs well and does not create a large payload
- If the evaluation of HTML payload is non performant, then the following approaches should be investigated in this order:
  1. Use current bundle mechanism:
    - (Kibana Jade template) to call API and generate the bundle which will be loaded during start-up
    - The JavaScript bundle produced will be of the following format: i18n_\<language>.bundle.js
    - Kibana loads all resource bundles on the client side after starting the single-page application
  2. Client side directly calls REST API and loads the JSON payload:
    - GET /i18n/translations ==> returns the English (or German or whatever) translations negotiated with the browser HTTP header “accept-language” priority list compared against the languages supported
    - GET /i18n/translationsForDefaultLocale ==> returns the default locale translations
- All approaches will decide the language to be served up by using the language deciding algorithm (from Phase 1)

Translation Identifiers Added to Kibana UI

Ids are added to the relevant view with Angular and HTML content
English (en) translation file is generated for the Ids defined

BEFORE (HTML)

    <div class="sidebar-list">
    …

    <div class="sidebar-list-header">
        <h5>Selected Fields</h5>
    </div>

AFTER (HTML)

    <div class="sidebar-list">
    …

    <div class="sidebar-list-header">
        <h5 translate="FIELDS_SELECTEDFIELDS"></h5>
    </div>

Tool for Verifying All Translations have Translatable Strings (Update)

Verify translation IDs in Angular templates
The keys in the key file(s) would then be checked against the language translation files registered

Translation Plugin Generator

Tool which generates a translation plugin
Localization engineer should only need to add translation file(s) within the plugin directory, refernce the file(s) in the 'uiExports' plugin extension, and add plugin to Kibana
All integration with Kibana works OOTB

Deliverable

Angular and React views where globalization is enabled showing the pattern to use
Verification tool updated to check translation keys in UI
A generator that localization engineers can use to produce language translations and integrate them in Kibana in an easy manner
Documentation relevant for the following stakeholders:
- those that want to enable translation in Kibana
- those that want to contribute translations to Kibana
- those that want to create a Kibana Plugin

Phase 3

Purpose

Add the translation identifiers and English translation strings for the existing Kibana views

Translation Identifiers Added to Kibana UI

Ids are added to all the relevant UI content (HTML, JS etc.)
English (en) translation file(s) are generated for the Ids defined

Deliverables

English translation bundles are available so that localization engineers can start generating translation plugins for different languages

Phase 4

Purpose

Contributing language packs

Handling Language Packs

Provide a pseudo language which will test the potential real estate of views for all language
Define core supported languages
Process for language pack builders:
- Creating packs
- Merging packs

Deliverables

Process for contributing language packs
Core language packs as supported by Kibana

Blockers

Open Questions/Issues

Translation of user data. TODO: requires a separate issue.
Server side set locale (ignore the client's requested locale)
Provide mechanism in UI for user to manually switch their language from list of supported languages
What about bi-directional language support? This affects language, charts and UI content support.
UI views should be able to handle different languages (This means when switching from one language to another (for example, English to German) that the look and feel is maintained) :
- Different languages can have variable content lengths. This means having adequate spacing to handle the strings in each language.
- Need to consider different fonts. Some web fonts won't support all possible languages.

History

This section has links to prior versions of this issue text.

Authors

Matt Bargar
Martin Hickey: @hickeyma
Scott Russell: @DTownSMR
Shikha Srivastava: @shikhasriva
Steven R. Loomis: @srl295

sunneyoy commented 6 years ago

Looking forward to the Chinese version.

udyr-f commented 6 years ago

Have you already supported Chinese? config/kibana.yml i18n.defaultLocale: "zh_CN" This configuration does not work, so I use this tool now. Kibana_Hanization

kimjoar commented 6 years ago

@sunmaolin Sorry, we currently only support English.

epixa commented 6 years ago

I'm going to close this issue out in favor of #17201. While that is a much newer issue than this one, it contains a lot of information about our current localization effort, and it's best if we don't fragment the discussion across multiple issues.

Many of the details in that other ticket are based on all the things we learned from this effort in particular, so thank you to the great folks at IBM and everyone else that has been involved in getting us to this point!

elastic / kibana