I18n messages

[brunosiqueira]

Issue type:

[x] question [ ] bug report [ ] feature request [ ] documentation issue

Database system/driver:

[ ] cordova [ ] mongodb [ ] mssql [ ] mysql / mariadb [ ] oracle [x] postgres [ ] sqlite [ ] sqljs [ ] websql

TypeORM version:

[ ] latest [ ] @next [x] 0.0.11

Is there a way for me to translate all the messages in typeorm into another language?

[pleerock]

there is no I18n inbox functionality yet, but we are very open to suggestions on design proposals of this feature.

[pleerock]

The first thing coming into the mind is to implement something like this:

// user defined interface and constant

export interface LocaleMap {
       en: string;
       es: string;
       cn: string;
       ru: string;
}

export const SupportedLocales = ["en", "es", "cn", "ru"];

@Entity()
export class Post {

     @PrimaryGeneratedColumn()
     id: number;

     @I18nColumn("name", { locales: SupportedLocales, eager: true, default: "en" }) 
     names: LocaleMap;

     name: string;

     // design notes:
     // selection will map column value into "name" property
     // locales array is optional
     // you can specify locale array in global connection options or not specify at all (any string key can be used)
     // you can specify eager: true to always load entity with all it names in find queries (in QB you'll need to add "addSelect" explicitly)
     // default is optional can be set in global options

}

Will store data in a following way:

id	name_en	name_es	name_cn	name_ru
1	...	...	...	...
2	...	...	...	...

Then, saving:

const post = new Post();
post.names = { "en": "...", "es": "...", "cn": "...", "ru": "..." };
await getRepository(Post).save(post);

And reading:

const posts = await getRepository(Post).find({ locale: "es" });
const posts = await getRepository(Post)
           .createQueryBuilder("post")
           .where("post.id > 1")
           .locale("ru")
           .getMany();

[brunosiqueira]

I don't think I would set the locale in the query rather than a setting while initializing typeorm. A more simple solution would be to:

1. Extract all the messages to an internationalization json file (e.g., locale/en_US.json).

   error: {
    unique: {
        value: 'the column $key is unique. Value is $value.'
    }
   }

2. Set a Messages manager class, where you could call all the messages from you code

   
   import {messages} from '../locale/messages.js';

messages.get('error.unique.value', {key: 'email', value: emailValue});

3. Then, you could pass a new locale file to the new instance respecting the same keys in another language:

error: { unique: { value: 'A coluna $key não permite valores duplicados. O valor é $value.' } }

4. If a value is not found in the new file, it falls back to the default one.
5. Regarding localizing tables and columns, I really like Ruby on Rails approach. Using an YML file with activerecord >> models >> attributes structure. And the framework takes care of the rest.

activerecord: models: user: Usuário profile: Perfil attributes: user: email: Email password: Senha password_confirmation: Confirmar Senha profile: name: Nome nickname: Apelido description: Sobre mim birth_date: Data de Nascimento gender: Sexo

[pleerock]

wait why your proposal contains internationalization of regular strings? Its not a problem to i18n-ze regular strings, its out of typeorm scope - you need a separate framework for this purpose. TypeORM's purpose on i18n is on how to save data in multi-language format.

[brunosiqueira]

If the ORM has pre-defined default error messages for field validations, than it would benefit a lot from internationalization. E.g., the error when the primary key is null: https://github.com/typeorm/typeorm/blob/master/src/error/PrimaryColumnCannotBeNullableError.ts Ideally, this.message would receive not a raw string as the message, but a key from an external file which would contain all the messages the ORM uses to communicate with developer or even the user. The same would be valid for unique field validation, not null validation, length validation, etc.

[pleerock]

Ideally, this.message would receive not a raw string as the message, but a key from an external file which would contain all the messages the ORM uses to communicate with developer or even the user.

There are no errors thrown by ORM that must be seen by users. Most of ORM errors are critical and must be fixed before running app in prod. Some of them can occur during lifetime of your prod app, but they must be caught and handled with a proper user messages.

The same would be valid for unique field validation, not null validation, length validation, etc.

ORM does not provide such validations. If you want validation you shall use validator library instead.

[michaelbromley]

I am thinking about this issue at the moment. I'm developing an ecommerce framework and users will be able to create products with localizations to any one of the IETF language tags. Therefore the suggestion above of having a column for each possible locale might prove unworkable - tables could easily end up 100s of columns wide.

This stack overflow question contains a good exploration of the issue, and I am currently inclined to adopt the accepted answer, i.e. for each entity with localizable fields (eg products), create a separate table for the localized strings (products_translations), which contains columns for each localizable string field and an additional column to specify the locale code. This would then support an arbitrary number of locales for each entity.

I'm not sure yet how I'd implement this with the current version of TypeORM, so I'd be really interested to see built-in support for this.

[pleerock]

@michaelbromley thank you very much for this wonderful link. I understand that issues you will have with multiple columns approach, but its the simplest possible solution. Implement it using another table is extremely hard to implement to cover all typeorm's edge case scenarios (and believe me you'll have a lot) and implementation will cost a months vs days.

But best of course if we will support both approaches. Anyway this is must have feature and feel free to contribute on any approach.

[michaelbromley]

@pleerock I understand if it is overly complex to implement my suggestion. I have very little idea about what is required internally in the TypeORM lib to do something like that.

Anyway, I have now implemented a solution in my app based on the idea of splitting an entity into 2 tables - the "base" entity and the "translations" entities which contain only those strings which should be localizable. I will share the details of this below, in case it proves useful to you or others looking to solve this problem:

Note: this solution requires TypeScript 2.8+ since it makes use of conditional types.

Types

First, we define some types which will help us enforce type safety across the various classes:

/**
 * This type should be used in any interfaces where the value is to be
 * localized into different languages.
 */
export type LocaleString = string & { _opaqueType: 'LocaleString' };

export type TranslatableKeys<T> = { [K in keyof T]: T[K] extends LocaleString ? K : never }[keyof T];

export type NonTranslateableKeys<T> = { [K in keyof T]: T[K] extends LocaleString ? never : K }[keyof T];

/**
 * Entities which have localizable string properties should implement this type.
 */
export type Translatable<T> =
    // Translatable must include all non-translatable keys of the interface
    { [K in NonTranslateableKeys<T>]: T[K] } &
        // Translatable must not include any translatable keys (these are instead handled by the Translation)
        {
            [K in TranslatableKeys<T>]?: never
        } & // Translatable must include a reference to all translations of the translatable keys
        { translations: Translation<T>[] };

/**
 * Translations of localizable entities should implement this type.
 */
export type Translation<T> =
    // Translation must include the languageCode and a reference to the base Translatable entity it is associated with
    {
        languageCode: string;
        base: Translatable<T>;
    } & // Translation must include all translatable keys as a string type
    { [K in TranslatableKeys<T>]: string };

Implementation

Now let's see how this would apply to a simple Product entity. First of all I define an interface which is the public-facing interface and hides away the fact that the entity is actually split across 2 tables:

export interface Product {
    id: number;
    name: LocaleString;
    description: LocaleString;
    image: string;
    createdAt: string;
    updatedAt: string;
}

Next we define the ProductEntity, which contains the non-localizable fields of the Product interface. The Translatable<Product> interface help to make sure we don't accidentally include any localizable fields here.

@Entity('product')
export class ProductEntity implements Translatable<Product> {
    @PrimaryGeneratedColumn() id: number;
    @Column() image: string;
    @CreateDateColumn() createdAt: string;
    @UpdateDateColumn() updatedAt: string;
    @OneToMany(type => ProductTranslationEntity, translation => translation.base)
    translations: ProductTranslationEntity[];
}

Finally we need to define the ProductTranslationEntity, which contains only the localizable string fields as well as a reference to the base entity and the languageCode.

@Entity('product_translation')
export class ProductTranslationEntity implements Translation<Product> {
    @PrimaryGeneratedColumn() id: number;
    @Column() languageCode: string;
    @Column() name: string;
    @Column() description: string;
    @ManyToOne(type => ProductEntity, base => base.translations)
    base: ProductEntity;
}

Usage

Let's say I want to get all Products. I use a query like this (note: I am new to TypeORM so this part may be non-optimum):

const products: Promise<Product[]> = this.manager.createQueryBuilder(ProductEntity, 'product')
            .leftJoinAndSelect('product.translations', 'product_translation')
            .where('product_translation.languageCode = :code', { code })
            .getMany()
            .then(result => result.map(productEntity => translate<Product>(productEntity)));

Where the translate() method is a utility which looks like this:

/**
 * Converts a Translatable entity into the public-facing entity by unwrapping
 * the translated strings from the first of the Translation entities.
 */
export function translate<T>(translatable: Translatable<T>): T {
    const translation = translatable.translations[0];

    const translated = { ...(translatable as any) };
    delete translated.translations;

    for (const [key, value] of Object.entries(translation)) {
        if (key !== 'languageCode' && key !== 'id') {
            translated[key] = value;
        }
    }
    return translated;
}

[pleerock]

approach you described does not have i18n functionality. You simply multiply rows, it works only in trivial system where you don't have relations. Once you'll have relations you get issues, duplication and lot of other problems.

[michaelbromley]

@pleerock true, my system is currently trivial as it is in the early stages of prototyping. I'll see how the above solution pans out as I build this out a bit more. If I get any other ideas I'll post them here. 👍

[mkosturkov]

My 50 cents on the topic.

First, I don't really think this feature belongs in an ORM.

That being said, how about storing translatable strings as JSON in their own fields? So, if we have a field description instead of adding columns or tables just store {"en": "Blah blah", "fr": "le blah le bluh"} in it.

Even if the DB does not support json, so what?

[pleerock]

@mkosturkov its okay for a very trivial operations. Just imagine you need to search something by name or use any other more or less advanced db operation.

[mkosturkov]

@pleerock I am not aware of any modern databases that do not support a JSON type of some sort and operating on it. However, even if the case is such, we are talking about translatable fields - that is most probably user entered text. So, what would be the complex operations, except for text searching, that may occur? I agree it's harder, but still doable.

[pleerock]

@mkosturkov sqlite doesn't :) Even if they all support JSON its obviously won't be effective solution. Working with JSON in any database isn't the most pretty thing to do. Working with your database will increase your project complexity.

[michaelbromley]

@pleerock @mkosturkov I've been following this issue since I posted my solution above. As an update, I've used a modified version of that pattern successfully in my current project. Right now I am testing on an ecommerce database with ~8k product variants, with each variant having translated facets & translated options, each of which is translated following the scheme above.

In short, I'd no longer consider it "trivial" since there are a number of related entities all with translations. The main problem from my perspective is that I have to manually apply the correct translation each time I do a select operation, but that's not too much trouble.

An example can be found here: https://github.com/vendure-ecommerce/vendure/tree/master/server/src/entity/product

[timojokinen]

@michaelbromley I think your solution is suitable for my project and I'm going to apply it, thanks! I was wondering if you ever implemented language fallback functionality and if yes, how?

[Luke265]

My experimental approach, which almost works like a plugin, but involves a lot of magic. Example: https://github.com/Luke265/typeorm-i18n/blob/master/samples/basic/index.ts

[kop7]

one more approach:

example:

LanguageEntity:

@Entity('language')
export class Language {
    @PrimaryGeneratedColumn() id: number;
    @Column() language: string;
    @ManyToMany(type => Product, product => product.id)
    products: Product[];
}

id	language
1	en
2	au
3	be

ProductEntity:

@Entity('product')
export class Product {
    @PrimaryGeneratedColumn() id: number;
    @Column() name: string;
    @Column() description: string;
    @ManyToMany(type => Language, language => language.id)
    @JoinTable({ name: 'product_language' })
    languages: Language[];
}

id	title	description
1	title_english	description_english
2	title_australia	description_australia
3	title_belgium	description_belgium

Table: product_language

productId	languageID
1	1
2	2

Query:

 const qb = this.productRepo.createQueryBuilder('p');
        qb.leftJoin('p.languages','pl');
        qb.where('pl.language = :lang',{lang: 'en'});
        return await qb.getMany();

Response:

 { 
 "id": 1,
  "name": "title_english",
  "description": "description_english"
 }

do you think that makes sense? what are the potential problems? @pleerock @michaelbromley

[imnotjames]

I'm going to be closing this.

Neither feature requested here belongs in the ORM from what I can understand.

The ORIGINAL feature as requested in the top of the issue was in relation to the error strings. The strings in our errors are for DEVELOPER USAGE ONLY. It is NOT suggested to expose the strings as written from our exceptions directly to users. If our error objects do not provide enough information for you to translate them, please open a new issue for that.

Separately, a feature was requested to support built in "multiple languages" in a field. if you're trying to build an internalization system on top of TypeORM, go hog wild. :) But that won't get merged in upstream to the core TypeORM. That's far too much business logic and not a feature that would be used enough to make sense from a maintenance perspective.

[HanMoeHtet]

I just released typeorm-translatable inspired by michaelbromley.