Investigate implementing an l20n-based localization library

[steveklabnik]

Issue by alexcrichton Wednesday May 28, 2014 at 18:15 GMT

For earlier discussion, see https://github.com/rust-lang/rust/issues/14495

This issue was labelled with: A-libs, P-high in the Rust repository

---

Part of this issue is only actionable depending on the outcome of https://github.com/rust-lang/rfcs/pull/93.

Localization is a very important feature for a language and is often a very difficult system to get right. Previous attempts with format! do not look like they're going to pan out, and it looks like it's possible to implement a high quality localization system purely in a library implementation.

The most likely candidate for being implemented is the l20n spec. This is a new standard coming out of Mozilla for the web which is still in some development, but it has seen deployment on Firefox OS and is planned to be deployed in Firefox. Other projects such as django are also looking into implementing this spec, so there's certainly precedent.

A rust implementation of the l20n spec would be a good solution for our localization efforts, but some finesse would be required to ensure that the library is as easily usable as the JS api itself.

Nominating, but I believe that this is not necessary for 1.0

cc @zbraniecki

[steveklabnik]

https://github.com/rust-lang/rust/issues/14495#issuecomment-45821815 is especially releavnt, as some work was being done on this already.

[simon-i1-h]

Cf. http://projectfluent.org/

[jibal]

"l20n" is a bizarre term. The usual terms are l10n, short for localization, which has 10 letters between the l and the n, i18n for internationalization, and g11n for globalization.

[White-Oak]

@jibal http://l20n.org/ It's a play of words. Citating https://wiki.mozilla.org/L20n L20n (L10n x 2) is the code name for a localization architecture taking existing approaches to L10n one step further

[jibal]

@White-Oak It's an ill-considered one. l10n, i18n, and g11n make sense and fit an established pattern ... l20n is just dumb marketing junk ... and I've actually seen Rustaceans misuse l20n as if it was the shorthand for "localization". And the l20n details seem to be quite an overengineered mess and a poor fit to Rust. Rust should go with gettext and other established approaches to i18n ... if l20n ever shows any success, Rust can consider adopting it. Unfortunately, Rust is way behind on the important issue of i18n and contains many roadblocks like format strings being statically parsed by the compiler. The chauvinistic inertia against i18n has been overcome by others; hopefully Rust will somehow do the same.

[Diggsey]

It seems weird to say Rust is "way behind" on this issue, when you can perfectly well use gettext-like approaches right now via libraries, but what languages have l10n built right into the language?

I think the premise of this issue is no longer true: we have many new language features since this issue was created, like procedural macros, which mean that l10n can be implemented as a crate without sacrificing syntax or usability. There's no need for it to be a language level feature.

If you think it should be done a particular way, implement it, publish it to crates.io, and maybe if it becomes popular it could join rust-lang-nursery, or be otherwise "blessed" in some way as the recommended way to do l10n. Starting with "we need to make language changes" seems entirely the wrong way to approach this issue.

[jibal]

I didn't even remotely say anything about making language changes. I did mention the problem with format! being implemented in the compiler and only taking static strings. And there's no support in the library for locales. And the compiler's messages cannot be localized. Rust is an ecosystem, and as an ecosystem it's clearly way behind. Saying "you can implement it and maybe if it becomes popular" doesn't change that, it just confirms it.

[Centril]

@Diggsey are there some notable crates you are aware of that you could post for everyone's benefit? If so, I think we can close out this issue as completed (closing issues on this repo makes me happy...).

[Diggsey]

@jibal this repo is specifically for changes to the Rust language, so this issue should be closed then, or at least re-purposed. The format! macro has limitations, but it's not intended for l10n to begin with, and seems an ill-fitting basis for l10n anyway. It is in no way "a roadblock to l10n".

A quick search of crates.io shows crates like fluent, amethyst_locale, rocket_i18n, etc. Being Rust, you are of course also free to call out to any existing C libraries providing localization.

A lack of localisation for the compiler's own error messages is definitely a problem Rust should try to solve, but is quite separate from localisation libraries existing in the ecosystem, and is a problem that few other compilers have yet solved.

[jibal]

"comes across as the ranting of a mad person, and is offensive to all the people who volunteer their time to improve Rust"

This is obviously in severe violation of the Rust code of conduct. I won't correspond with you any further.

I will note for others that the name of this RFC is "Investigate implementing an l20n-based localization library", which is not about changes to the language. From https://github.com/rust-lang/rfcs, this repo is not just for changes to the language ... language changes, library changes, and compiler changes are all listed as subjects for RFCs.

Also, the relationship of format! to i18n and l10n is clear and well understood and the consequences of the limitation have been remarked on for years, with hopes that the restriction could be removed. "it's not intended for l10n to begin with" doesn't make sense ... i18n is a property of the whole system -- the ability to adapt it to various cultures -- and l10n is the crafting of a culturally specific dataset for an i18n framework. Other language ecosystems allow for l10n of the primary mechanism for generating text messages, and a comment by Graydon Hoare said that it was intended for format! but people ran out of time.

[Centril]

Given the list given by @Diggsey, I think we can close this as being completed. For further discussion about localization, I suggest trying either http://users.rust-lang.org/ or http://internals.rust-lang.org/.

[jibal]

I would like to note that the name of this RFC is "Investigate implementing an l20n-based localization library", which is not about changes to the language. From https://github.com/rust-lang/rfcs, this repo is not just for changes to the language ... language changes, library changes, and compiler changes are all listed as subjects for RFCs.

Also, the relationship of format! to i18n and l10n is clear and well understood and the consequences of the limitation have been remarked on for years, with hopes that the restriction could be removed. Note that i18n is a property of the whole system -- the ability to adapt it to various cultures -- and l10n is the crafting of a culturally specific dataset for an i18n framework. Other language ecosystems allow for l10n of the primary mechanism for generating text messages, and a comment by Graydon Hoare said that it was intended for format! but people ran out of time.

[zbraniecki]

Hi @jibal

l20n is just dumb marketing junk

No it's not. It's a play on words. Whether it is a good play on words or not, is up to you. But as one of the authors of the project, I promise you it was not meant as a marketing term :)

We changed the name of the project several years ago to Fluent - https://projectfluent.org/

Rust should go with gettext and other established approaches to i18n ...

I'm not sure how much experience you have with l10n, but here's our teams take on shortcomings of gettext - https://github.com/projectfluent/fluent/wiki/Fluent-vs-gettext

I'd recommend Rust to either go with a bit dated, but high quality system Unicode MessageFormat, or Fluent ( https://github.com/projectfluent/fluent-rs )

Here's on differences between them: https://github.com/projectfluent/fluent/wiki/Fluent-and-ICU-MessageFormat

[jibal]

@zbraniecki

Whether it is a good play on words or not, is up to you.

I already stated my opinion that it was ill-considered, and why. I think you're taking "marketing" too literally ... taking a widely used term like l10n that follows a specific pattern and morphing it into l20n because that particular implementation is supposedly twice as good or advanced or whatever is "marketing" in a broad sense of the term, and just doesn't look good from the outside. I have no problem with "fluent".

Thanks for the links.

[mbrubeck]

Moderator note: Everyone, please keep the focus on technical discussion of current or future options for Rust code.

As this is a fairly outdated issue, it would probably be best to stop discussion here and instead draft an RFC for any specific proposed changes, or to ask questions in new or existing threads in internals.rust-lang.org (for things implemented in the language and standard libraries) or users.rust-lang.org (for thing related to external crates and tools).

[jibal]

@mbrubeck

Check. Thanks. Over and out.