Create an HTML Helper and Tag Helper that enables rendering localized string from ResourceManager

[DamianEdwards]

A LocTagHelper and HtmlHelper.Loc HTML helper method that allow specification of a localized resource string to use.

Usage

as element:

<loc id="" [param-*=""]>
    The default locale content string {*}
</loc>

as attribute:

<ANY asp-loc-id="" [asp-loc-param-*=""]>
        The default locale content string {*}
</ANY>

as HTML helper:

@Html.Loc(id: "", default: "", formatParams: new { ... })

Arguments

Attribute Name	Type	Details
id	string	The ID that uniquely identifies the resource string
param-*	string	Values to substitute into the resource string at runtime

Resource Key Generation

Details to come...

Examples

title key = "/Views/Home/Index.cshtml>input#Field1[title]" content key = "/Views/Home/Index.cshtml>button#submit" title key = "/Views/Home/Index.cshtml>button#submit[title]" content key = "/Views/Home/Index.cshtml>button#submit" title key = "/Views/Home/Index.cshtml>button#submit[title]"

Example

Source CSHTML

<div>
    <h1 asp-loc-id="WelcomeHeading">Welcome!</h1>
    <p asp-loc-id="WelcomeText">
        Some <em>welcome</em> text.
        <!-- Using loc tag helper -->
        <loc id="foo" param-count="@Model.ShoppingCart.Items.Count">You have {count} items</loc>

        <!-- Using loc HTML helper method -->
        @Html.Loc(id: "MyId", default: "You have {count} items", formatParams: new { count = Model.ShoppingCart.Items.Count })
    </p>
    <form>
        <input asp-for="Field1" title="This is field 1" asp-loc-id-title="Field1Key" />
        <span asp-validator-for="Field1"></span>

        <button type="submit" title="Click me" asp-loc-id="submit">Submit</button>

        <button type="submit" title="Click me"
                asp-loc-id="submit"
                asp-loc-param-OrderId="@Model.ShoppingCart.Items.Count">
            Submit Order {OrderId:G}
        </button>
    </form>
</div>

[stanislavromanov]

This would generate html that is not semantic? Shouldn't you use data-* for custom properties?

[yishaigalatzer]

@DamianEdwards lets triage.

[yishaigalatzer]

@stanislavromanov what you are seeing is the cshtml the html will not have any of these attributes or tags, only the localized string.

[yishaigalatzer]

@DamianEdwards lets set up a design meeting so we can get loc supported up and down the stack

[glen-84]

1. This is mostly (completely?) translation, and not other things like number/date formatting, etc., so I think it would be better to give it a name that makes this obvious (t, tr, trans, translate, etc.)
2. "loc" also seems like a shortening of "locale" (and not just localization), so "loc-id" is like "locale identifier", which you may expect to be something like "en-US". If you only want to support resource files, then maybe something like "res" would make more sense.
3. The HTML helper syntax is rather verbose for something that will be used a lot within views. I would suggest using something like @T() instead. This is used by the Orchard CMS, if I'm not mistaken (meaning possible conflicts), but I don't think that that should influence the decision too much (the helper could be disabled on demand or you could use @TR(), etc.). @Res() maybe if this is just for resources.
4. You mentioned that you don't really like the @using() syntax for capturing content. Have you thought about supporting new syntax for this, such as:

@T() {
    This is
    a paragraph
    of text
}

@Html.Form(...) {
    <div>
        @Html.TextBoxFor(m => m.UserName, new { @class = "form-control" })
    </div>
}

... where everything between the braces is available to the HTML helper?

---

The following points may not be directly related to this issue, but I'm not sure where else to put them. (I will move them if you tell me where)

(1) Regarding the selection of the culture/locale, will you be doing something similar to this (as middleware)? i.e. strategy-based selection. I think Orchard also has something like this. Basically, you have a prioritized list of strategies (implementing ICultureSelector or ICultureSelectionStrategy, for example). You iterate over each of them, calling a method such as "Select" or "GetCulture", which either returns a Culture or null if it cannot be determined using that strategy. The first strategy to return a Culture will break the loop. After this, a method (f.e. "Update" or "SetCulture") will be called on each strategy, which will allow it to update the HTTP request/response (it could remove a query string param, set a cookie, change the host, change the URI path, etc.). If the method makes any such changes, the user will be redirected (on GET requests only). Examples of strategies would be:

QueryStringStrategy           (?culture=en)
HttpAcceptLanguageStrategy    (Accept-Language: "en-GB,en;q=0.8,de;q=0.6,nl-NL;q=0.4,nl;q=0.2")
HostNameStrategy              (example.de or de.example.com)
CookieStrategy                (Cookie: culture=en;)
UriPathStrategy               (example.com/en)
etc.                          (load from user preference in the database, etc.)

Each strategy would have its own set of options (I won't go into the details). You would also set a list of supported cultures for your application.

(2) Is any of the following planned (with regard to translation)?:

• Alternative translation "data-sources", f.e. loading translation data from a database, or Gettext, CSV, etc.
• Plural translations. It seems like there are at least two systems for representing plural rules/forms:

CLDR Language Plural Rules (plural categories / mnemonic tags) Gettext PO Plural Forms (plural form / header)

Note that a number of systems only allow one word in a message to be pluralized, so you can't properly translate a message like "He ate {0} apple(s) and {1} banana(s)". This is not good.

See: https://github.com/wikimedia/jquery.i18n#plurals See: https://developers.facebook.com/docs/opengraph/guides/internationalization#plurals

• Gender-based translations.

See: http://stackoverflow.com/questions/4272933/what-is-the-best-way-to-translate-to-a-language-with-genders-in-rails See: https://github.com/wikimedia/jquery.i18n#gender

• Grammar-based translations.

See: https://github.com/wikimedia/jquery.i18n#grammar

• Use from places other than views (translating in the controller [probably shouldn't happen much], translating model display names & validation messages, etc.)
• The ability to opt into events like "MessageTranslated" and "MissingTranslation".
• Message documentation. The ability to comment on or explain the context of a message.

See: https://github.com/wikimedia/jquery.i18n#message-documentation See: https://docs.djangoproject.com/en/1.7/topics/i18n/translation/#contextual-markers See: https://www.gnu.org/software/gettext/manual/html_node/Contexts.html#Contexts

• Text domains / namespaces. Grouping messages into logical "domains".

Create a domain per "area", have a "default" domain which may or may not override translations in areas, etc.

• Other localized resources, f.e. images. For this it might be easiest to just do something like: <img src="@{"x_" + CurrentCulture + ".png"}" />

From my limited research, I see two options for the message format, both of which should support multiple plurals/gender/grammar etc.:

1. SmartFormat.NET
2. ICU MessageFormat
   • A .NET implementation can be found here.

So putting this all together, it may look something like this (in a Razor view):

<p>
    @T() {
        {gender, select, male {He} female {She}} ate {count, plural, one {1 apple} other {# apples}}.
        Line 2.
    }
</p>

gender and count could come from the current scope, or you could use numeric placeholders, like:

<p>
    @T(gender, count) {
        {0, select, male {He} female {She}} ate {1, plural, one {1 apple} other {# apples}}.
    }
</p>

Without using the "content capture" syntax:

<p>
    @T("{gender, select, male {He} female {She}} ate {count, plural, one {1 apple} other {# apples}}.")
</p>

<p>
    @T("{0, select, male {He} female {She}} ate {1, plural, one {1 apple} other {# apples}}.", gender, count)
</p>

There is also number and date/time formatting, etc., which I haven't included above.

Powerful localization features could give ASP.net a competitive advantage.

[DamianEdwards]

@glen-84 thank you very much for the fantastically detailed and thoughtful suggestions. It's really appreciated. I hear you on the naming and we'll certainly consider alternatives.

Adding support for new syntax in Razor is obviously a fairly high bar and we're beyond the time for 1.0.0 where we could reasonably do that, so we'll support the standard HTML helper (@foo()) and Tag Helper approach for now (<foo>, <div asp-foo="">).

We're fleshing out these plans right now for 1.0.0, but it will include at a minimum:

• a middleware for setting the culture on the request (from header, cookie, and custom delegate);
• a new localization abstraction API (ILocalizer) for easy retrieval of locale-specific strings including substitution of values with culture specific formatting (which would be backed by ResourceManager out of the box but of course you could implement your own to do whatever you want);
• updates to MVC to support the new API, including a view specific ILocalizer and updates to the validation and metadata model provider system to support localization of strings from validation and model metadata attributes
  • we might also support locale specific views (e.g. Index.fr.cshtml) but this may ship as a sample to start out

We won't be able to cover all your suggestions for 1.0.0 but of course we can improve our support in subsequent releases. Like I said, we'll support standard .NET ResourceManager/resx in v1.0.0 but it will be behind a new interface so you can substitute it in your app easily with a different system that supports whatever features you like. I don't see anything in your suggestions that our current plans for the contract won't support.

I hope to have a sample project published in the following week that shows what we're planning for 1.0.0. I'll update this issue with a link when it lands.

[DamianEdwards]

@glen-84 the sample site containing a functional (but not at all complete and likely very buggy) prototype of much of our plans is now at https://github.com/DamianEdwards/i18nStarterWeb

Note that it's using very recent bits that may not play well with your version of Visual Studio, but it should be enough to give you an idea.

[glen-84]

Thank you very much ...

You're welcome, and thank you for reading. :smile:

Adding support for new syntax in Razor is obviously a fairly high bar and we're beyond the time for 1.0.0 ...

Understood. Do you think something like this may be considered for the next release of Razor (post 4.0)? It's no secret that I'm not a big fan of tag helpers (and would love them to be implemented as a plug-in), so it would be nice to have a more code-focused alternative for things like the cache helper, etc.

A middleware for setting the culture on the request (from header, cookie, and custom delegate);

How will you handle prioritization? For example if both a cookie is set, as well as a header or delegate, which takes priority? Would using a priority queue and a culture selection interface take a lot of time to implement? (for v1 you could leave out the update/set-locale side of the process). The other benefit here is that users could easily create other "strategies" (although I guess they could write their own middleware if necessary).

A new localization abstraction API (ILocalizer) for easy retrieval of locale-specific strings including substitution of values with culture specific formatting ...

This sounds (and looks) good so far, unfortunately I get about 36 errors when trying to build the solution, so I can't play around with it much.

Updates to MVC to support the new API, including a view specific ILocalizer and updates to the validation and metadata model provider system ...

• Won't there be a lot of duplication if you have one resource file per view?
• Will it be possible to configure validation messages without using attributes? (defaults in addition to per-type overrides) I think that putting the validation messages in the models will get messy quite quickly.

We might also support locale specific views ...

I've seen this type of thing before, but I think that (a) there will be loads of duplication, and (b) it will get messy [especially with more than just a few supported locales]. There may be performance implications as well, with file existence checks, fallbacks, etc. I would definitely keep this as "disconnected" as possible, I don't think that many people will use it (but I could be wrong).

We won't be able to cover all your suggestions for 1.0.0 ...

That's expected. The main purpose of my research and suggestions was to give you an idea of the type of functionality that would be required in a "perfect-world" scenario. The important thing is that you have this information in the back of your mind while you are designing the APIs. If the interfaces are flexible enough, implementations can be added/updated later (or contributed).

I hope to have a sample project published in the following week

You're ahead of schedule! :smile:

Some initial feedback / questions:

1. When you use locString["Home"], does that string represent the "id" or the "default"? I'm assuming the latter. How would you specify the ID?
2. Have you considered adding an option to quietly handle formatting exceptions? I think it's possible that the resources could come from an external location and contain formatting errors. Maybe the default message could be used if there is a formatting exception (opt-in)?
3. Is it necessary to have ResourceManagerLocalizer in addition to ResourceManagerWithCultureLocalizer? Is it not possible to just have:

ResourceManagerLocalizer(ResourceManager resourceManager, CultureInfo culture = null)

(or two constructors)

That's it for now, I might add more once I'm able to run the application.

PS. Can you let South Africa win the World Cup this time? :laughing:

[Anderman]

First, I realy like the new MVC and C#

I am a little bit afraid you put a lot of effort on some code what nobody will use (for example the asp-fallback-href-exclude)

I do think a lot is already done. Create a good fundamentals and some basic implementation and the community will do the rest

The @T() function of @glen-84 is possible. WIth beta5 you can use the using static MyClass. And if I use SmartFormat. Then I can make something like

Smart.Format(TranslationClass.Translate("{Gender:he|she} ate {Apple.Count:choose(0|1):nothing|one apple|{} apples},CurrentCulture),model);

I can make a my own class that will first do the translate and then do a smart.format.

public class MyClass{
   private readonly Culture _currentCulture;
   public static T(string s, object model)
   {
      Smart.Format(TranslationClass.Translate(s,CurrentCulture),model);
   }
}

Of Course there is lot more needed.

The properties of the model must sometimes translated too. This can be solved by adding a [translationvalue] attribute to a property. A Fork of Smart.Format maybe needed for this. Or a MS version that first translate and then format

If you have more then a few pages you will need a database for the translation.

• Translation should be added if missing and the user is not a admin or something
• Management of translation for user with a special role
• example of the translated template with a example model

The database is very important if you develop an application. Because messages are always added, changed or removed in the application live cycle. For translators, message must be seen in the context of the rest of the page. I have seen applications where you can right click to translate some messages

For removed message we could use a lastTranslatedate or something. Or we could do somethiing smart with mocking the translation class when testing

Some Notes: I don't like this kind of attributes in my model

        [Required]
        [StringLength(100, ErrorMessageResourceType = typeof(loc), ErrorMessageResourceName = "TheMustBeAtLeastCharactersLong", MinimumLength = 6)]
        [DataType(DataType.Password)]
        [Display(ResourceType = typeof(loc), Name = "Password")]
        public string Password { get; set; }

I prefer something like this

        [Required]
        [StringLength(100, ErrorMessage = "The {Display} Must Be At Least {maxstringlength} Characters Long", MinimumLength = 6)]
        [DataType(DataType.Password)]
        [Display(Name = "Password")]
        public string Password { get; set; }

And very important: We need that all strings , error message and exception are send by the DI to FirstTranslateAndThenFormat class.

[IDisposable]

The downside off putting the ErrorMessage directly in the text is that you can't easily localize it without extensive code changes. As a resource, you can just replace one or more resource .dll files.

[Anderman]

I see your point but that why I prefer a database for translation instead of resource file. Then you can change the translation on the fly.

If all message go through the translation module then it always possible to translated. I like the idea that we can plug in our own Translation and format system. Resource files should be one of the options

The current implementation of resource file is not so flexible.

• First you don't know all the messages that the system can send. These message can come from the MVC, OS or some other system.
• The validation rules of the user and password could only intercepted after the message was formatted. The email adres someone@somewhere.com is already used

See for example this one https://github.com/abergs/OwinFriendlyExceptions or http://geekswithblogs.net/shaunxu/archive/2012/09/04/localization-in-asp.net-mvc-ndash-upgraded.aspx. Solving only a small part of the multi-language problem

[mcquiggd]

I write a lot of Globalized applications and I agree that having a mechanism for determining culture, specifying culture, and obtaining localized strings would be a major selling point for the framework.

There are some excellent comments on this thread as to functionality that is required, suggestions for future enhancements etc. I will add my own checklist (that does contain some duplication of comments above, and some extras).

Firstly, what needs to be localized in a typical application:

1. Text displayed in Views.
2. Text displayed via JavaScript.
3. Text within code, e.g. to compose a Message returned via WebAPI, Exceptions or Logging.
4. Model / View Model Properties used to build forms.
5. Model Validation messages.
6. Route URLs (SEO friendly urls, slugs).
7. Support for SPAs!
   • 1-5 are basic functionality that should be present in the Framework.
   • 6 might be handled by exposing an extension point for the developer to plug-in their own implementation.
   • For Views, it is necessary to have the concept of Global texts (e.g. header, footer), and route specific texts (Home\Contact).
   • For Model Properties, texts these should be fully qualified by their namespace.
   • For Model Validation, they should support the standard data annotation attributes, and allow custom messages.
   • JavaScript libraries exist that can be dynamically loaded with i18n support (e.g. a Calendar); but for messages displayed by JavaScript (e.g. Thank you for subscribing to our newsletter), should be retrieved from the same source of localized text, e.g. one common repository.
   • From experience, Resource files quickly become unwieldy, and a database with subsequent auto-expiring caching (e.g. Redis) is optimal. Whatever interface is described to accept an injected Localization Repository should allow for caching options.
   • Support for right to left languages can usually be handled by adding a culture to a CSS declaration as described above for images.
   • Jonas Gauffin has some comments http://blog.gauffin.org/2013/01/better-support-for-localization-in-asp-net/, and a fairly good implementation of localiziation in his MVCContrib project. It is worth checking out; most of the above functionality is covered, and more, such as hashed keys, scanning View Models for display text that requires translation, indicating text that has not been translated for a culture, etc. This last point is much better than a tag helper that leaves text visible in the default language - [fr-FR] Translations.Home_Welcome is much more obvious than seeing default text.
   • Nadeem Afana also has an series of posts http://afana.me/post/aspnet-mvc-internationalization-store-strings-in-database-or-xml.aspx that provide an MCV 5 solution.
   • Rick Strahl also has a library that may give inspiration - http://weblog.west-wind.com/posts/2015/Mar/17/ASPNET-MVC-Localization-and-WestwindGlobalization-for-Db-Resources.
   • Strong typing, e.g. T(Translations.Home_Welcome) is useful but sometimes impractical. But, IMHO there are way too many magic strings in ASP.Net 5.

In terms of detection and specification of culture, these include:

1. Browser accept headers.
2. Cookies.
3. Database stored preferences.
4. Query strings (IMHO ugly solution).
5. URL segments / Route values; e.g.en.microsoft.com/, microsoft.com/en-GB/
   • https://owinglobalization.codeplex.com/ is a good implementation of most of the above, including an optional extension of Identity to include a preference, and being able to specify it a culture is invariant.

Just my thoughts.

I have looked at the original i18nStarterWeb project, but I am more pleased with the https://github.com/aspnet/Localization sample, that seems to be heading in the right direction. Above everything else, add extension points and support DI for our own custom implementations...

[Eilon]

@DamianEdwards is this beta8 or backlog?

[djanosik]

I've created two simple libraries for Localization and Validation that we use in large applications. They cover some of the points mentioned by @mcquiggd:

1. Resources can be accessed anywhere in the application, eg. Resources.Get("Category:Item").
2. A dynamically generated JS file (~/assets/resources.js) can be used to access values from a dictionary for the selected culture.
3. See point 1.
4. All display names are localizable. It will look for the following keys in resources:

Annotations:{lastNamespacePart}_{modelName}_{propertyName}
Annotations:{modelName}_{propertyName}
Annotations:{propertyName}

5. Messages for client or server-side validation are also localizable. The following keys are used:

Annotations:{lastNamespacePart}_{modelName}_{propertyName}_{validatorName}
Annotations:{modelName}_{propertyName}_{validatorName}
Annotations:{propertyName}_{validatorName}
Annotations:Error_{validatorName}

It's quite easy to localize messages that come from DataAnnotations attributes. Everything you need is to implement your own IValidationMetadataProvider. However, it is not possible to localize all messages. Some of them are created in the ModelStateDictionary and there is no way to jump in.

[rynowak]

@DamianEdwards - does this issue still make sense in the world of 2017?

[DamianEdwards]

I still think the Tag Helper might be useful. I'm going to create one over at https://github.com/DamianEdwards/TagHelperPack/ so we might get some indication of whether it's a good fit to add to MVC by default.

[danroth27]

No plans to add this to ASP.NET Core.