Should the interface provided by the data model and the interface provided to translators be tightly or loosely coupled?

[eemeli]

This is a design principle issue identified within the smaller-group huddle working on the data model.

The question is a reformulation of what was presented to the group during last week's meeting (#158) as the second question, and which wasn't directly discussed much during the meeting. It's not strictly a data-model internal question, but a question that's driving alternative approaches to data model design. This issue should be considered as a blocker for the data model.

If we determine that the coupling should be tight, then that allows and requires for the interface layers between the data model and e.g. the XLIFF 2 representation of the same to be relatively simple and direct. This would impose relatively strict limits on the scope of the data model, as it requires anything representable in the data model to have a decent representation in current interfaces provided to translators.

If we determine that the coupling should be loose, then the interface layers built on top of the data model get potentially more complicated, as they must negotiate a match between two different worlds that do not necessarily completely align in their structures. This would allow for a data model that may represent a wider set of messages than is available with current interfaces provided to translators.

It's important to note that the loose-coupling approach would not prevent us from providing an XLIFF 2 interface that's equally direct and equally powerful to the interface provided by tight coupling, provided that certain restrictions are imposed on the messages being represented in the data model. The difference is that it would allow for the representability at the data model level of messages that are not supported by said interface due to their structural complexity.

Resolving this question one way or the other would rather strongly inform our current and future work, and it should be sufficiently abstract to not let us get stuck in the details of the data model.

[mihnita]

I really don't like framing it in terms of coupling.

It has nothing to do with coupling, and I don't understand what does it mean in this context. I also find it unclear what "interface" means here.

We develop something used to handle strings for UI (not relevant if they are rendered on a screen, or read aloud). Should be able to translate them, group them per language, download on demand, support all languages, support advanced linguistic features, etc. So designing something that works well for people using this (translators) is not coupling, is design for the function.

If I design a car with 12 pedals, it is bad design, not "coupling with the fact that people have 2 legs"

[eemeli]

To clarify the terms:

• By "interface provided by the data model" I mean the representation provided directly by the data model. For example, it's what a MF1 -> MF2 converter tool would need to output to represent a message.
• By "interface provided to translators" I mean the representation of MF2 messages used by external translation/localization tooling. For example, this could be an XLIFF 2 file.

I use the term "interface" rather than e.g. "representation" to underline the fact that the bindings can be two-way, and that we cannot presume that any one format would always be considered the canonical one by our users.

To continue with the above examples, I at least would consider the following an entirely valid use case:

1. Messages are originally held as MF1 source, e.g. within Java properties files.
2. Tool X parses the MF1 source of each messages into an MF2 data model representation of the same message.
3. Tool Y transforms the data model representation into an XLIFF 2 representation of the same.
4. The XLIFF 2 representation is used to provide a translation of the messages.
5. Tool Y' transforms the XLIFF 2 representation of the translated messages into their MF2 data model representation.
6. Tool X' stringifies the MF2 data model representation as MF1 source messages.
7. The translated messages are stored on disk in a MF1 Java properties file.

Here, both the data model and XLIFF 2 are interfaces that allow for the MF1 source messages to be worked with more effectively.

In the context of that example, the question of this issue is relevant when determining what happens if the original source of step 1 is using some structure that isn't supported by the XLIFF 2 representation. I'm pretty sure that won't happen with pure MF1, but bear with me. In this situation, which of the following should happen?

1. Tool X fails, because the data model cannot represent the message.
2. Tool Y fails, because the XLIFF 2 representation does not support this data structure.

"Failure" here can of course mean that a sub-optimal representation of the message is used, rather than just erroring out.

As a proponent of loose coupling, I would say that in this case the data model should be designed in a way that it makes it more likely that any such source structure could be representable with it, and that the failure should happen in Tool Y rather than Tool X.

Finally, in order to have something a bit more concrete, here's an example of an MF1 extension for ranges that might trigger this situation (with apologies if I'm mangling the Dutch):

{[start, end], selectrange,
  one {van {start} tot {end} dag}
  other {van {start} tot {end} dagen}
}

That could of course be reformulated to use a single range variable and a range formatter, but the point here isn't to change this original source, it's to determine if we can represent such a message in the data model, even if it were not representable in our chosen XLIFF 2 representation.

[dminor]

Maybe it is not the best analogy, but I have been thinking about this in terms of soundness and completeness. In a sound data model, it is impossible to represent invalid data. In a complete data model, all valid data can be represented.

Since it's unlikely we can come up with a model that is both sound and complete, we have to make tradeoffs. Is it more important to prevent representing invalid data, even if that means that we also can't represent some valid data, or is it more important to be able to represent all valid data, even if that means having to deal with invalid data in our tooling?

I think this is a valid question. To me, a simpler model means we think soundness is more important, and a more expressive model means we think completeness is more important.

[echeran]

I have been thinking about this in terms of soundness and completeness. In a sound data model, it is impossible to represent invalid data. In a complete data model, all valid data can be represented. ... To me, a simpler model means we think soundness is more important, and a more expressive model means we think completeness is more important.

I agree with @dminor , with one subtle change in wording. Similar to above, I see the tradeoff described here is an instance of the inverse relationship between simplicity and power (although in this explanation, "generality" is used instead of simplicity, and "execution" is used instead of power). I prefer the word simplicity because I see things like soundness and generality as consequences of simplicity (and it's something we can talk about more precisely). But I'm not opinionated on choosing among complete / expressive / powerful.

[eemeli]

While we do need to determine the soundness/completeness/simplicity of the data model, we need to first determine whether that ought to be directly linked with the interface that translators will have with the same messages.

If the linkage is direct, these aspects must align with that translator interface, and our decisions about these aspects must take that into account.

If the linkage is not necessarily direct, these aspects may align, but we are also able to make different choices, should that prove beneficial for other reasons.

Until we answer the question posed in the title of this issue, we can't conclusively resolve the entirely valid questions posed above by @dminor and @echeran.

[dminor]

I was thinking about data validity specifically from the point of view of the translation tool when I wrote my earlier comment. To use @eemeli 's example above, to me soundness implies that Tool X can fail, completeness that Tool Y can fail, and I was hoping to find some terminology that we could use here instead of "coupling" given @mihnita 's concerns about that word.

I completely agree that there are broader issues around the soundness/completeness/simplicity of the data model that we should discuss, but it wasn't my intention to turn this issue into that broader discussion.

[mihnita]

I think it is really tiresome to keep "chasing" things around in so many places. We have a document with the main questions. And slides. And two documents with arguments for one / other approach. And now this issue, opened as the main question, but does not exist in the "questions document". But if we answer this one, we solve everything... (although I don't think we agreed anywhere that this is the core question)

---

OK, let's take a shot at it....

what happens if the original source of step 1 is using some structure that isn't supported by the XLIFF 2 representation I'm pretty sure that won't happen with pure MF1,

It does happen. This is why I have that proposal to extend XLIFF 2.x

1. Tool X fails, because the data model cannot represent the message.
2. Tool Y fails, because the XLIFF 2 representation does not support this data structure.

At what step are we talking about? Tool X is invoked at step 2 and 6, tool Y at step 3 and 5.

Bypassing that, there are a couple of red flags (for me at least):

I don't care about round-tripping MF1 through MF2 data model. Yes, nice to have, probably possible. But that is not the goal of this project. This is not why we design this data model. Making is the central

2. Tool X parses the MF1 source of each messages into an MF2 data model representation of the same message.

Should not fail. A few exceptions: features considered bad (numeric placeholders, patterns for dates), choice.

All features present in MF1 should be supported in MF2 data model. The features that are not supported directly (internal selectors) are algoritmicaly converted to message level selector. The result of that is still compatible with MF1. If this fails, we failed. We designed something that does not even support what the 15 years old MF supported.

---

3. Tool Y transforms the data model representation into an XLIFF 2 representation of the same.

With the caveat that the XLIFF 2 needs some extensions.

5. Tool Y' transforms the XLIFF 2 representation of the translated messages into their MF2 data model representation.
6. Tool X' stringifies the MF2 data model representation as MF1 source messages.

I think that with the "flexible model" step 6 can fail. Because translators are allowed to add extra things that are not supported by MF1. And I think that with the "strict model" both steps would succeed, because the translator can "tweak" only the features allowed by the developers. And if the developers didn't use any MF2 features, the translator can't add them.

We can make the "flexible model" work if "throw that over the fence", out of the model, and into "validation / lint" And here we get to what I was saying: you need that validation to run as part of the localization tooling. Which means that the "registry" used for that validation should be somehow portable, through XLIFF 2, and to the localization tools, and to the TM tools. And that adds A LOT of complexity.

---

The example:

{[start, end], selectrange,
  one {van {start} tot {end} dag}
  other {van {start} tot {end} dagen}
}

Can we choose something that solves a real linguistic problem? From what I understand the message should deal with something like "from 3 to 4 days"

On what are we making the plural decision? On start, or on end? Why? Are we sure this is linguistically correct in all languages? Is there a language where we need to look at both start and end? And if yes, are sure the decision can be represented by using a single "case", and we don't need all combinations? ([one, one], [one, other])

And what happens if start == end? We get "from 3 to 3 days", which is bad. This can't be fixed with the current English example.

We try to support linguistic needs, and we want to help developers do the right thing.

That is why I think we should try to use real use cases, not cases that are artificially construed to make a point or another (same reordering the adjectives example)

That could of course be reformulated to use a single range variable and a range formatter, but the point here isn't to change this original source

Why not?

This is like arguing 30 years ago that we should change the cars to be safer: "That could of course be solved by asking the driver to use a seatbelt, but the point here isn't to change the original situation, where drivers don't do that"

Or "we should make strcpy safer, but without asking the developer to add the size of the target buffer"

We introduce artificial restrictions, and then we "show" that it can only be "solved" with one model. By passing the burden from the developer (1 person) to the translators (40-80 people, non-technical).

[eemeli]

1. Tool X fails, because the data model cannot represent the message.
2. Tool Y fails, because the XLIFF 2 representation does not support this data structure.

At what step are we talking about? Tool X is invoked at step 2 and 6, tool Y at step 3 and 5.

I meant steps 2 & 3; steps 5 & 6 use the X' and Y' transforms.

I don't care about round-tripping MF1 through MF2 data model. Yes, nice to have, probably possible. But that is not the goal of this project.

I would think that being able to use the MF2 data model as a translation layer (pun intended) between translation systems is a pretty significant goal. Don't we want to provide an AST-type target that any message markup or syntax can be parsed into, and then build tooling on top of that? Because we could.

1. Tool Y' transforms the XLIFF 2 representation of the translated messages into their MF2 data model representation.
2. Tool X' stringifies the MF2 data model representation as MF1 source messages.

I think that with the "flexible model" step 6 can fail. Because translators are allowed to add extra things that are not supported by MF1. And I think that with the "strict model" both steps would succeed, because the translator can "tweak" only the features allowed by the developers. And if the developers didn't use any MF2 features, the translator can't add them.

We can make step 6 impossible to fail, if the Y transform to XLIFF 2 limits what's available to translators. That's sort of my whole point here.

The example:

{[start, end], selectrange,
  one {van {start} tot {end} dag}
  other {van {start} tot {end} dagen}
}

Can we choose something that solves a real linguistic problem? From what I understand the message should deal with something like "from 3 to 4 days"

On what are we making the plural decision? On start, or on end? Why? Are we sure this is linguistically correct in all languages? Is there a language where we need to look at both start and end? And if yes, are sure the decision can be represented by using a single "case", and we don't need all combinations? ([one, one], [one, other])

And what happens if start == end? We get "from 3 to 3 days", which is bad. This can't be fixed with the current English example.

CLDR now includes data on range plurals, which operates on the plural categories of the start and end values. In the absence of other evidence, I at least would trust that source as linguistically correct.

[aphillips]

Is this issue still useful? Let's close if not. Otherwise, it's tagged for Future

[aphillips]

Reopen if desired for Future