Type/position based matches and `data-l10n-pos` (Item 4 and Item 5)

[zbraniecki]

Axel and Flod reviewed the proposed list of items and listed item 4 and 5 and their concern points that they'd like to discuss further.

From my understanding Flod is particularly concerned about Item 5, while both Axel and me believe that we need to consider 5 if we decide to add 4.

My position is based on the following logic: If we allow for Item 4, then we enable developers to write DOM Fragments that without Item 5 cannot be reordered by localizers.

Flod pointed out that so far we have not encountered cased where localizers want to reorder, but unless we decide that we believe localizers will not likely ever want to reorder, we should imho design our API to be extensible in the future to allow for that. Otherwise we may lock ourselves into an API that we'll have to refactor in a backwards incompatible way.

I'm opening this issue to discuss the merits of this proposal.

[Pike]

4. Overlay elements based on type and position

As a developer, I don't want to have to use data-l10n-name for all functional child elements.

As a localizer, I want to edit the most streamlined version of the markup possible, without attributes which add cruft.

<div data-l10n-id="key1">
  <ul>
    <li></li>
    <li><button id="..." onclick="..."></button></li>
    <li></li>
  </ul>
</div>

key1 = This is a list of errors with
  <ul>
    <li>Bullet 1</li>
    <li><button>Submit</button></li>
    <li>Bullet 3</li>
  </ul>

Proposed behavior: Allow for the source structure to be replicated in the translation and auto-match elements on each "level" based on their type and position.

5. Allow translations to overlay elements in different order

As a localizer, I want to be able to reorder two child elements on the same nesting level.

<div data-l10n-id="key1">
  <a href="https://www.mozilla.org"/>
  <a href="https://www.firefox.com"/>
</div>

key1 = Open <a data-l10n-pos="2">Firefox</a>, which is a <a data-l10n-pos="1">Mozilla</a> product.

Proposed behavior: Localization can optionally use the data-l10n-pos attribute to explicitly match a different elements than it would get assigned based on the order in DOM.

[Pike]

Item 5 is similar to what we've done with printf in the past. en-US would have some %S does %S, and localizers could do %2$S is done by %1$S.

That has worked out so badly, that we completely stopped allowing that in Firefox code. In any printf string with more than one positional argument, all arguments need to be explicitly numbered. The codepaths for implicit ordering still work, but you get an 🦉 if you try to use 'em.

That's the main reason why Fluent only has named external variables, pretty much regardless of the developer convenience, or bundling API. If your programming language has a problem with dicts, deal with it ;-). Also why the data-l10n-args uses the crutch of serializing the args to json.

From an l10n echo system POV, I'm strongly against Item 5.

CC @flodolo

[flodolo]

key1 = Open <a data-l10n-pos="2">Firefox</a>, which is a <a data-l10n-pos="1">Mozilla</a> product.

My main problem with this approach is that, as a localizer, I'm already setting the order of the elements in my translations. You're basically asking me to look at the source, realize that there are nodes that change order, then explicitly call them out (and make sure I match them correctly).

It's a level of complexity that we shouldn't add for localizers. Few will understand or be aware of it, it can break way too easily, and without ways for us to realize. With data-l10n-name, we could at least warn if one of them suddenly disappear.

key1 = Open <a data-l10n-pos="1">Firefox</a>, which is a <a data-l10n-pos="2">Mozilla</a> product.

[zbraniecki]

Thanks for the feedback!

I'll provide counter-arguments so that we have the complete picture:

Having name as the only way to match source to localization means that every element that is meant to be matches has to have a name.

That turns

Click on <img/> to continue.

into

Click on <img data-l10n-name="logo"/> to continue.

which brings the number of characters in the string from 28 to 50, almost double, and all the new characters are not translatable.

and

<div data-l10n-id="key1">
  <img src="logo.png"/>
</div>

into

<div data-l10n-id="key1">
  <img src="logo.png" data-l10n-name="logo"/>
</div>

both of which, I'd argue, bring more noise and make the code less readable.

But the problem scales badly. With the introduction of more complex structures like nested ones we're moving from:

<ul data-l10n-id="key1">
  <li><img src="logo1.png"/></li>
  <li><img src="logo2.png"/></li>
  <li><img src="logo3.png"/></li>
</ul>

and

key1 =
  <li>This is <img/> line 1</li>
  <li>This is <img/> line 2</li>
  <li>This is <img/> line 3</li>

to:

<ul data-l10n-id="key1">
  <li data-l10n-name="li1"><img data-l10n-name="logo1" src="logo1.png"/></li>
  <li data-l10n-name="li2"><img data-l10n-name="logo2" src="logo1.png"/></li>
  <li data-l10n-name="li3"><img data-l10n-name="logo3" src="logo1.png"/></li>
</ul>

and

key1 =
  <li data-l10n-name="li1">This is <img data-l10n-name="logo1"/> line 1</li>
  <li data-l10n-name="li2">This is <img data-l10n-name="logo2"/> line 2</li>
  <li data-l10n-name="li3">This is <img data-l10n-name="logo3"/> line 3</li>

and that moves from 98 characters to 231 - increase by 140% - and all of the new characters are, once again, completely meaningless to 99% of localizers and bring just noise.

Since almost no localization wants to reverse order of anything, ever, it seems like a bad design to make everyone, always, pay for an off-chance that someone, somewhere will want to reverse the order.

To give a comparison point - out of ~150 locales in CLDR, the List Format pattern is reversed in one locale - ur-IN.

Now, that brings me to Item 4 - we should allow to skip the data-l10n-name and automatch based on the element's name and position, automatically.

Now, once you get there, you will likely, somewhere, at some point, hear from someone that they do, in fact, need to reverse. It may not happen today, as my gut feeling is that in none of the strings we have, in none of the locales, we have a case like that, but some day it may.

That's where item 5 comes in. This one locale, in this one case, could then use data-l10n-pos to reverse it.

My concern with Pike's and flod's position is that they seem to argue about the data-l10n-pos as if it was an API that localizers would commonly use, and make mistakes with it in result (Axel's point on $1). I believe that localizers would almost never use it, and thus will not make mistakes with it, and this one string for this one locale that does need it, could be rather easily verified to match.

With data-l10n-name, we could at least warn if one of them suddenly disappear.

We can also warn them without data-l10n-name if there's an element in localization that couldn't be matched to any element in the source (or reverse).

[zbraniecki]

There's also a milder version of the proposal possible:

It we stick to Item 7, we could recognize if there's only one element of a given type in a given node, and allow for automatching on it. That would alleviate some of the cost, without introducing the ordering problem.

The l10n would then look like this:

Click on <img/> to continue.

key1 =
  <li data-l10n-name="li1">This is <img/> line 1</li>
  <li data-l10n-name="li2">This is <img/> line 2</li>
  <li data-l10n-name="li3">This is <img/> line 3</li>

[flodolo]

I believe that localizers would almost never use it

I don't think looking at the CLDR list order is relevant. Here's an example from Firefox (I didn't look further, but I'm sure there are more, unfortunately it's really hard to tell for DTDs since we glue strings together for them).

Search %1$S for “%2$S”

These are plain placeables, so they could just be moved around in Fluent. But if someone decides to use a <span> to style them, you have an example of node reordering.

I agree with you that having too many data-l10n-name doesn't help readability either. But at least it's straightforward from a localizer perspective.

[stasm]

The l10n would then look like this:

Click on <img/> to continue.

key1 =
    <li data-l10n-name="li1">This is <img/> line 1</li>
    <li data-l10n-name="li2">This is <img/> line 2</li>
    <li data-l10n-name="li3">This is <img/> line 3</li>

FWIW, I don't see good use-cases for reordering structural block elements like li in this example. I recommend only considering inline elements like img, span and a.

[zbraniecki]

FWIW, I don't see good use-cases for reordering structural block elements like li in this example. I recommend only considering inline elements like img, span and a.

I'm afraid of a slipper-slope reality (not fallacy ;)) here. If we allow block elements to be matched based on the structure (so, without data-l10n-name), then someone, at some point, will do:

key = Select <menulist/> to open <menulist/>.

and you'll have to resolve that, and at that point you'll either get back to requiring data-l10n-name on block elements, or allow for data-l10n-pos on block elements. That's especially relevant in the light of upcoming web components.

[Pike]

I agree that trying to solve this on a block vs inline level won't scale. Custom elements break that, and also for html, block vs inline or block-inline can be easily changed by CSS. Both of which are invisible to Fluent. And I'd rather keep that that way.

Iff concise writing is an issue, I'd rather have us revisit the idea of using elements of of the target markup language. The react overlays are pretty terse in the Fluent files, using the node name as parameter name. The custom element names do obfuscate a bit what shows in the end, but <input> vs <img> probably doesn't convey a lot to less-technical localizers.

I'm not sure that that's the right path, though. I also don't have an ad-hoc answer on how to implement that outside of react.

Some high-level concerns of mine:

I'd really like to take the story around deep markup with various interacting elements out of the picture in the conversation for DOM overlays. If a project puts a complete dialog into a Fluent message, someone shot themselves in the foot, and they shall bleed.

I am also concerned about implications that not-so-100% strings have. The main driver here is that w/out tooling support in pontoon, we're not going to have 100% correct strings. And I wouldn't know how to cram tooling support for something complex into Pontoon this year.

My old hat still says that whatever isn't supported on pontoon will lead to errors. I'd much rather have us have something with small implications that's fault tolerant. That would possibly allow small cheap incremental steps to implement in Pontoon.

I'm still having some "it's OK to use p, ul and li here" in my head, and then do a best effort in the DOM sanitizing logic.

[stasm]

I agree that trying to solve this on a block vs inline level won't scale. Custom elements break that, and also for html, block vs inline or block-inline can be easily changed by CSS. Both of which are invisible to Fluent. And I'd rather keep that that way.

When I mentioned block and inline elements, I meant the formal content models defined by HTML. I now see that they're called differently: flow content and phrasing content, respectively. I see how the distinction between doesn't apply well to what we're discussing here. Thanks to both @zbraniecki and @Pike for providing examples which made it clear.

I'm still having some "it's OK to use p, ul and li here" in my head, and then do a best effort in the DOM sanitizing logic.

My current thinking is still far away from the whitelist approach, although I appreciate its simplicity and explicitness. Any alternative should be at least as simple as the whitelist approach.

I want to go back to the idea of block vs. inline elements again, but this time, let's redefine what these terms mean. I'm not using them to mean flow and phrasing from the HTML spec anymore. Instead:

• Inline element is any element which developers want to put inside of a run of text (or sentence).
• Block elements are all other elements.

Inline elements require a hint to order them correctly inside of a continuous run of text. This solves the use-cases of multiple <a>, <img>, <select> and <menulist> existing in a single sentence. The name of the attribute which provides the hint could (currently: data-l10n-name) could be changed to clearly call out the fact that the element is intended to be surrounded by text. data-l10n-inline could be one possibility.

Block elements, OTOH, create the scaffolding around which the translation is built. The are <ul>, <p>, <div> and more. They don't require the attribute hint because they cannot be moved. Block elements in the translation are matched against the block elements in the source based on their path in the fragment tree.