Localization & Active Locales

[HeyJoel]

Localization is possible but requires changes to be made directly in the database. This area hasn't been looked at much and ideally we need to spend a bit of time working out the exact requirements of a flexible cms locale system.

• Ideally we'd have a way to define active locales in code
• The routing would be flexible (although we need to investigate the different ways sites might want to do this)
• We need to create a localized sample site
• We need to ensure that we test for situations where a locale is enabled and then disabled again with resources attached.

Any input here as to what might be needed by a locale system are welcome

[HeyJoel]

This area needs working on as part of the asp.net core migration. For now the active locale is determined by the current thread culture and available locales are still set in the db. The current thread culture can be set using the built in request localization middleware, but you need to define the supported culture collection to match your db localization settings.

TODO:

• Create a localization sample site so we can play around with the scope of localization in practice
• Work out a configuration strategy that is simple, extensible and in keeping with asp.net core.
• Implement a RequestCultureProvider for our standard url culture format e.g. 'cofoundry.org/en-GB/'
• Create a pluggable path formatter to allow customization of localized urls to be used in GetAllPageRoutesQueryHandler.CreateFullPath and IPagePathHelper

Test startup code:

var supportedCultures = new[]
{
    new CultureInfo("en-US"),
    new CultureInfo("en-GB"),
    new CultureInfo("es")
};

var options = new RequestLocalizationOptions()
{
    DefaultRequestCulture = new RequestCulture("en-US", "en-US"),
    SupportedCultures = supportedCultures,
    SupportedUICultures = supportedCultures
};
app
    .UseRequestLocalization(options)
    .UseCofoundry();

[HeyJoel]

Some additional info about why the culture list is restricted by default on this issue

[snovak7]

Possibility to hook a localisation to a domain name, let's say domain.com is english, domain.de is german.

[HeyJoel]

The test startup code earlier in the issue comments does not set the RequestCultureProvider for the current Cofoundry locale url pattern. This lead to issue #240, the sample in the response might be of use to readers here:

public void Configure(IApplicationBuilder app)
{
    var defaultCulture = new RequestCulture("en");
    var supportedCultures = new[]
    {
            defaultCulture.Culture,
            new CultureInfo("es")
    };

    var options = new RequestLocalizationOptions()
    {
        DefaultRequestCulture = defaultCulture,
        SupportedCultures = supportedCultures,
        SupportedUICultures = supportedCultures,
    };

    options.RequestCultureProviders.Insert(0, new CustomRequestCultureProvider(context =>
    {
        foreach (var supportedCulture in supportedCultures)
        {
            var segementToTest = '/' + supportedCulture.IetfLanguageTag;
            if (context.Request.Path.StartsWithSegments(segementToTest, StringComparison.OrdinalIgnoreCase))
            {
                var result = new ProviderCultureResult(supportedCulture.IetfLanguageTag);
                return Task.FromResult(result);
            }
        }

        return Task.FromResult(new ProviderCultureResult(defaultCulture.Culture.IetfLanguageTag));
    }));

    app
        .UseRequestLocalization(options)
        .UseCofoundry();
}

[snovak7]

By the example provided, is it possible to have localization done over different domains? .com, .de, .se, etc...

[HeyJoel]

You should be able to set the locale by domain using that method, but some parts of Cofoundry may not work as expected, particularly pages where the route is still built using a set format i.e. /en-gb/mypage. I don't have an example setup to test with yet but looking at the code it looks like it shouldn't effect dynamic routing but rather just the url rendered in the admin panel or via @Cofoundry.Routing.Page(route).

When we implement this feature we will look into routing locale via domain name.

One aspect to think about would be how to handle writing out urls, which would typically be relative by default, but if you're routing locales by domain and writing a link to a page with a different locale then you'd need to include the domain. Additionally IContentRouteLibrary.ToAbsolute would have to take the dynamic domain name into consideration.

[hishamco]

@HeyJoel what can kind of help do you need to close this?

FYI I already did many localization stuff including: ASP.NET Core Localization APIs, SimplCommerce and Orchard Core

[HeyJoel]

Hi @hishamco

I think a large part of the work here is in designing a flexible but simple to implement locale system. There is a legacy implementation in Cofoundry that isn't documented because it was developed to meet the needs of two specific websites prior to the .NET Core conversion and hasn't been through a thorough design and exploration process. It kind-of works with the above test code if you like to live life dangerously, but we don't recommend using it.

To start with it would be good to explore the requirements for this feature and any edge cases that may need to be handled. The above requirement for localisation based on a domain name is a good example of case that we hadn't considered. I haven't looked at this area recently, but off the top of my head here are some points to think about:

• The original implementation only supported urls in the format 'cofoundry.org/en-GB/', what other formats should be supported and how flexible does this need to be?
• We currently have cultures set as active in the database, this would probably best be done in code to save querying the db so often
• What entities should support localization? Currently this is pages and custom entities, but what about directories? blocks or entity data model properties? Do users need a locale property?
• What should happen when a locale is removed but has attached data?
• What is the editing experience? Do content editors tend to work on one localized site at a time (i.e. should we be able to filter globally to one locale)?
• In what ways can we help content editors with localizing content?
• How do others support this? What works, what doesn't?
• What other questions should we be asking?

Any help in contributing information here is most welcome!

We'll also need to build out a localized sample project to get a feel for how the implementation will work, and then obviously there's the development of the feature and migration of existing data.

Not every feature needs to be developed at once, but It's quite a fundamental feature that touches most areas of the system, so I'd like to make sure we design it to be robust and flexible.

[hishamco]

Perhaps this is a big one and we can break it to many pieces as you mentioned before, but I'd like to share my comments regarding your thought and questions above

The original implementation only supported urls in the format 'cofoundry.org/en-GB/', what other formats should be supported and how flexible does this need to be?

Url or routing localization could be set as default culture request provider if we do so, but I think we should support the other out of the box providers: cookie, query string, .. etc

We currently have cultures set as active in the database, this would probably best be done in code to save querying the db so often

Saving default culture and supported cultures in the database is a good option to feed the localization middleware. We already did this in SimplCommerce and coming soon in OrchardCore, regarding save querying the db this should be done once when the application starts after that it can read from settings or that data that is retrieved before

What entities should support localization? Currently this is pages and custom entities, but what about directories? blocks or entity data model properties? Do users need a locale property?

I need to look to the supported entities and how localization can fit nicely with them, similarly directories .. etc

What should happen when a locale is removed but has attached data?

I 'll defer this until the above point is becomes clear

What is the editing experience? Do content editors tend to work on one localized site at a time (i.e. should we be able to filter globally to one locale)?

IMHO the content editor can choose the locale to localize the current content (Content Localization)

In what ways can we help content editors with localizing content?

By providing a way to translate and localize the content though the admin panel

How do others support this? What works, what doesn't?

Please clarify this one

What other questions should we be asking?

There will be more questions during the localization journey 😄

The conclusion is:

1- Localization Management

• Default Culture & Supported Cultures

• Supported CultureRequestProvider(s)

• Underlying Localization Resources Store

• Culture Picker / Language Switcher

2- Content Localization

• Manage Localized Contents

• Underlying Localization Resources Store

• Translation Support

• Localization Permissions

• RTL Support

[HeyJoel]

Additional requirements taken from gitter chat:

...I have cases like when you have more domains, and my local language has .si domain, but other locale/language is .com, and german is under .com/de ...my idea would be that there is some kind of table which has columns: domain, prefix, alias (slug, url)

Also:

an example would be that we did a site for a food product which had 8 locale variations. For us some of the tricky aspects were that some of the designs for the locales had to be very different and in some parts required very specific templates that were locale specific. For the product nutritional information the format was very specific to a locale for legal reasons which made it impossible to content manage - this had to be hard coded and updated by the dev team.

For routing we just used the format example.com/en-gb, but we linked out to some sites that were managed by other local agencies under different domains e.g. example.co.uk.

So for this site each locale had a different set of pages e.g. /en-au/what-we-make/, /fr-be/nos-produits/. I've not worked on a site that used the same pages with simple translation of content regions but I presume it does happen.

[hishamco]

My aim from the beginning is to add some value to project, but while you didn't accept PRs, I 'll leave my comments for the history ..