[DRAFT—DO NOT MERGE/REVIEW] Add members for localization

[christianliebel]

• [ ] Clarify: Do we want to go with the pluralized form of the properties (e.g., name --> names)? What happens if we would localize icons (icons --> ?)? Or should we instead use a suffix like name_localized?
• [ ] Add algorithm for translatable members
• [ ] Adjust algorithm for shortcuts
• [ ] Add an example
• [ ] Explain the concept

---

Closes #???

This change (choose at least one, delete ones that don't apply):

• Breaks existing normative behavior (please add label "breaking")
• Adds new normative requirements
• Adds new normative recommendations or optional items
• Makes editorial changes (changes informative sections, or changes normative sections without changing behavior)
• Is a "chore" (metadata, formatting, fixing warnings, etc).

Implementation commitment (delete if not making normative changes):

• [ ] WebKit (https://bugs.webkit.org)
• [ ] Chromium (https://bugs.chromium.org/)
• [ ] Gecko (http://bugzilla.mozilla.org)

If change is normative, and it adds or changes a member:

• [ ] updated JSON schema

Commit message:

(Fill in. If making normative changes, describe exactly what the behavioral difference will be.)

Person merging, please make sure that commits are squashed with one of the following as a commit message prefix:

• chore:
• editorial:
• BREAKING CHANGE:
• And use none if it's a normative change

---

Preview | Diff

[aarongustafson]

I'd prefer adding the "_localized" suffix across the board as it makes it explicit what the member's purpose is.

[marcoscaceres]

Yeah, I guess it does make sense to make it "_localized" as this can only be used for that.

[dmurph]

From editors meeting:

• We want to also localize the icons, but because it already has an s that makes it a little weird
• Due to that, let's do *_localized for all of these fields.
• icons_localized won't have any triplets, we'll just resuse the parsing algorithm for icons (same json structure). But, like the others, it will be lang-string dictionary.
• These can apply as well within the shortcuts item, to be able to localize those.

TPAC discussion here

[marcoscaceres]

Instead of defining new members, let's instead define *_localized member pattern that is either a "text localizable member" or an "image-resource localizable member" (i.e., icons or for shortcuts). That way we don't need to define the algorithms over and over again.

[marcoscaceres]

We already have defined localizable members so we can already reuse that.

[marcoscaceres]

Here is what an shortcuts member might look like with some _localized members sprinkled in:

{
  "shortcuts": [
    {
      "name": "Play Later",
      "name_localized": {
        "fr": "Écouter plus tard"
      },
      "description": "View the list of podcasts you saved for later",
      "description_localized": {
        "fr": { "lang": "en", "dir": "ltr", "value": "English description because that's part of our brand." },
      },
      "url": "/play-later",
      "icons": [
        {
          "src": "/icons/play-later.svg",
          "type": "image/svg+xml"
        }
      ],
      "icons_localized": {
        "fr": [
          {
            "src": "/icons/fr/play-later.svg",
            "type": "image/svg+xml"
          }
        ]
      } 
    },
    {
      "name": "Subscriptions",
      "description": "View the list of podcasts you listen to",
      "description_localized": {
        "fr": "Consultez la liste des podcasts que vous écoutez."
      },
      "url": "/subscriptions?sort=desc"
    }
  ]
}

Note: updated to include a triple example.

[marcoscaceres]

Question is if we should allow localizing various URLs... that gets a bit messy in places, but could be doable.

[christianliebel]

@marcoscaceres @dmurph Thanks. Let's also localize urls. I'll resume working on this soon.

[mgiuca]

LTTP... thanks for working on this Christian!

Let's also localize urls. I'll resume working on this soon.

Wait, why are we localizing URLs? That seems undesirable to me. It means that shortcuts can go to different places depending on the language, and we need to update functionality that may be cached (that isn't just string/icon data). This may add non-trivial implementation complexity (it would require a deeper analysis to understand to what degree).

Would the use case for this be that you can have URLs with different ?lang= query parameters to match the user's language? I think I'd prefer just letting the website use the Accept-Language HTTP feature for this.

[christianliebel]

Wait, why are we localizing URLs? That seems undesirable to me.

Maybe we can also loop in @aphillips to see whether it makes sense to localize URLs.

[aphillips]

Icons, graphics, or remote content (help pages, for example) are sometimes varied by locale or by region (with locale serving as a poor proxy for region). This might be done, for example, because the icon contains some text or because a graphic shows a culturally-linked image (personal images, national costume, post box shapes, etc. etc.) that the user wishes to localize. Or it might be because functionality or defaults differ (sorting based on pronunciation instead of name for Chinese, for example)

We don't know why the user might want to localize the icon or shortcut (or whatever).

I agree that this can be abused and there might be reasons not to allow some fields to be localized, although I'd probably thinks about health warnings first?

[aarongustafson]

I think I'd prefer just letting the website use the Accept-Language HTTP feature for this.

Probably the most elegant, but it’s not within the realm of possibility for a lot of orgs and site types (thinking static sites, for example).

Would the use case for this be that you can have URLs with different ?lang= query parameters to match the user's language?

See also MDN style links where the language code is embedded in the URL path.

Icons, graphics, or remote content (help pages, for example) are sometimes varied by locale or by region (with locale serving as a poor proxy for region). This might be done, for example, because the icon contains some text or because a graphic shows a culturally-linked image (personal images, national costume, post box shapes, etc. etc.) that the user wishes to localize. Or it might be because functionality or defaults differ (sorting based on pronunciation instead of name for Chinese, for example)

We don't know why the user might want to localize the icon or shortcut (or whatever).

Agree on all of these.

[benfrancis]

Just one data point, but as a past precedent the similar Web of Things (WoT) Thing Description specification landed on titles and descriptions members of Thing (each of type MultiLanguage) in addition to title and description for this use case. This is the case for both Thing Description 1.0 (W3C Recommendation) and Thing Description 1.1 (W3C Proposed Recommendation).

I personally don't like that solution because I prefer using HTTP content negotiation with an Accept-Language header as per the suggestion in the current Web Application Manifest Working Draft, rather than creating an extremely verbose manifest with (theoretically) up to thousands of different languages. However, as has been pointed out it's not always possible to use content negotiation (e.g. on static site hosting like GitHub Pages). The Thing Description specification therefore offers both as alternative approaches.

If consistency between W3C specifications is considered important, then names, short_names and descriptions would make sense. That doesn't work for icons, but that member is already different because it's an array of ImageResources that the user agent can select from. Language could potentially just be another criteria for selecting an image.

I note that in HTML the <link> element has a hreflang attribute, so presumably <link rel="icon" href="/icons/fr/play-later.svg" hreflang="fr"> is valid (though likely currently ignored by user agents). For the (slightly unusual) case of localising app icons, an equivalent might be to add a lang member to ImageResource.

Example:

{
  "lang": "en",
  "dir": "ltr",
  "name": "Super Racer 3000",
  "names": {
    "fr-FR": "Super Coureur 3000",
    "es-ES": "Súper Corredor 3000"
  },
  "short_name": "Racer3K",
  "short_names": {
    "fr-FR": "Coureur3K",
    "es-ES": "Corredor3K"
  },
  "icons": [
    {
      "src": "icon.png",
      "sizes": "64x64",
      "type": "image/png"
    },
    {
      "src": "icon-fr.png",
      "sizes": "64x64",
      "type": "image/png",
      "lang": "fr-FR"
    },
    {
      "src": "icon-es.png",
      "sizes": "64x64",
      "type": "image/png",
      "lang": "es-ES"
    }
  ],
  "scope": "/",
  "id": "superracer",
  "start_url": "/start.html",
  "display": "fullscreen",
  "orientation": "landscape",
  "theme_color": "aliceblue",
  "background_color": "red"
}

Note that ImageResource also has a label member which could then also be localised this way, if a localised accessible description of the icon is needed!

One question: How does dir interact with the localised members? Can it safely be derived from language? The Thing Description specification has a lot to say on that topic, which I can't say I fully understand, but they use the Strings on the Web: Language and Direction Metadata W3C Note for guidance.

Hope this helps.

[mgiuca]

Perhaps we should clarify what "localize URLs" means.

Are we talking about:

• Localizing all fields that are URLs? (e.g. start_url, scope, potential future ones like the home scope of tabbed apps).
• Localizing all fields called "url", like the ones in icons and shortcuts?
• Something else?

I think we generally agree that we need to localize icons, but we're doing that at the icons level, not the url within icons (i.e. icons_localized with a local version of each icon dict, not icons with the icon dict including a url_localized. Allowing both of these creates two ways to do it which isn't ideal.)

Maybe it makes sense for URLs like shortcuts to be able to change based on language, but really the point of this initiative (I thought) was to be able to localize your app's metadata that gets displayed at the OS level, like name and icon, not to solve the problem of localizing all the content in the app. (If you can't configure your server to serve content based on headers, you can still make your service worker return localized content based on Accept-Language.) An app that relies purely on the manifest URLs to display content in the user's language is likely to be quite brittle.

I think we could run into major headaches if we allow scope to be localized. And by extension, start_url. (e.g. scope must be a superset of start_url - what happens if that's true in some languages but not others?)

I prefer if we just start with name, short_name, description and icons and go from there as needed.

How does dir interact with the localised members? Can it safely be derived from language?

This was discussed at TPAC (search for "dir") - unfortunately when I asked this question, the answer was not recorded ("?"). From memory, @aphillips pointed out that we may not know the language's direction because languages are not set in stone. I think the overwhelmingly common case will be a known language, which means we should be able to derive dir from lang in all practical cases (and probably default to ltr if we don't recognize the language - the overwhelming majority of languages are LTR). We should have the dir member for being explicit, but I think in 99.9% of cases the site should not need to specify dir as it can be derived from language.

[marcoscaceres]

Supportive of what @mgiuca said above... let's start small and go from there (and definitely let's not have multiple ways of doing the same thing, specially with URLs). And yes, let's keep the localizable members restricted to a small set (including image objects, not members within those objects).

[aphillips]

@mgiuca noted:

From memory, @aphillips pointed out that we may not know the language's direction because languages are not set in stone. I think the overwhelmingly common case will be a known language, which means we should be able to derive dir from lang in all practical cases (and probably default to ltr if we don't recognize the language - the overwhelming majority of languages are LTR). We should have the dir member for being explicit, but I think in 99.9% of cases the site should not need to specify dir as it can be derived from language.

Your specification should not derive direction from language unless there is no other alternative. This is not because languages are mutuable.

You may permit an item that lacks separate direction metadata to attempt to use the language to estimate the direction or to act as a hint, but this should not be the default way of doing it. In fact, I18N recommends making the direction auto when the dir is not present at the item or document-default level instead of using directional estimation based on language. We explicitly recommend using auto instead of ltr as the default, since an unlabeled string that starts with a strongly RTL character is probably trying to tell you something 😉.

We have extensive guidance in https://www.w3.org/TR/string-meta/ and we're working on an update to our guidance about manifests here (I hope to land this PR on Thursday) which you may find useful here.

[dmurph]

Manifest Working session notes:

This seems to be the concluded format (copied from Marcos's comment above) with triple examples:

{
  ...
  "dir": "ltr",
  "lang": "en-x-marcos",
  ...
  "shortcuts": [
    {
      "name": "Play Later",
      "name_localized": {
        "fr": "Écouter plus tard"
      },
      "description": "View the list of podcasts you saved for later",
      "description_localized": {
         "en":  { "value": "My App, hey!", "dir": "ltr", "lang": "fr"},
         "en-GB": { "value": "My App, eh wut?", "dir": "ltr"},
         "fr": "string",
         "ar": { "value": "...", "dir": "rtl" }
      },
      "url": "/play-later",
      "icons": [
        {
          "src": "/icons/play-later.svg",
          "type": "image/svg+xml"
        }
      ],
      "icons_localized": {
        "fr": [
          {
            "src": "/icons/fr/play-later.svg",
            "type": "image/svg+xml"
          }
        ]
      } 
    },
    {
      "name": "Subscriptions",
      "description": "View the list of podcasts you listen to",
      "description_localized": {
        "fr": "Consultez la liste des podcasts que vous écoutez."
      },
      "url": "/subscriptions?sort=desc"
    }
  ],
   ...
}

TPAC notes from this are here

[tomayac]

I suppose the stray "lang": "fr" in your comment before isn't intended:

"en":  { "value": "My App, hey!", "dir": "ltr", "lang": "fr"},

I could edit your comment directly, but wanted to make sure it's indeed a copy/paste first.