language variations as separate intents

[polx]

In the process of working on the translation of the raw intents of David F presented last Thursday, which took me less than 2h (but not including unicode symbols), I realized that the speak-aloud column in French may be richer than in English and that many more languages is going to trigger many more variations.

This issue is to discuss how much intents can cope for this expectation. Clearly handcrafted enrichments of intents are in a position to do it in many cases. Should the predefined intents' keywords ("core intents") simply ignore the variations?

[polx]

Examples in French:

• A ⊂ B: (A is included in B): A est inclus dans B. However:

  • if you know that A is a surface then A est incluse dans B (A is feminine).
  • If you write X, Y ⊂ B: then it should be pronnounced X Y sont inclus dans B or X Y sont incluses dans B.

• with just the same symbol (it could be most others): the "subjonctif" variation may be needed: Afin que A ⊂ B (So that A ⊂ B) should be pronounced Afin que A soit inclus dans B (est has become soit)

I know that some of this concerns English (eg. the plural). The problem behind this is really that we don't know how many variations are going to appear...

Is it a solution to allow "wildcards of a keyword" so that new language variations are possible to input but are still connected to their base?

Is it a well known problem for AT writer that such a refinement is not fully accessible? I suppose so.

[dginev]

One position is that:

• the function of the intent Core annotation is only to ground the specific concept (subset or proper-subset in the example above), and attach any arguments.
• the language synthesis task is left to specific implementers to instrument, based on:
  • the requested language (e.g. lang attribute),
  • the annotated name of the Core concept,
  • the annotated arguments
  • the underlying presentation tree.
• (optional) the Open list could offer additional help with known aliases and sample speech hints.

And the usual justification for this minimalist position is that it keeps the annotation burden small, while still allowing enough scaffolding for symbolic translation to be possible (if not always perfect).

---

For the example, adding an isa annotation to A that marks it as a surface could be a useful translation hint - but would require implementers to consult isa to infer gender in languages that require that.

Knowing to use the right verb forms for singular and plural nouns seems to be strictly in the domain of the implementer - intent already makes it possible to count the exact number of arguments.

I think it is a known issue that symbolic approaches to language translation quickly become large complex systems (e.g. language grammars), when they increase the amount of supported vocabulary, variation and syntax. I can testify that this also happens to grammars which try to model the full variability in math syntax. That said, neural translation approaches (NMT) starting with annotated MathML+Intent can directly leverage the annotations as well, and learn the language variation from a set of examples. They are also large complex systems, but at least we don't have to build them by hand. In any case, either approach is possible on a MathML+Intent expression, and either approach can have difficulties in arriving at a flawless translation on previously unseen expressions.

[NSoiffer]

I'll start with: gender in languages is hard for me to wrap my head around, but I do realize that a word may change based on the words around it or what it implicitly refers to. That can be hard to deal with and I suspect that several systems, such as the one Murray implemented in MS products, just punt on that. There is definitely merit in getting something in user's hands that is understandable (is it?) vs. trying to come up with a perfect solution and having nothing to use.

In your example, it seems that knowing that A is a set of female mathematician results in different speech than if you know A is a set of male mathematicians. Is that correct? If so, I suspect that tagging it with ISA won't help much because understanding the gender of what will be essentially a "random" phrase. Pulling the gender from a "random" phrase is sure to be beyond what AT is capable of knowing. We could consider adding a gender component to ISA, but what else would need to be added. Seems like a rabbit hole as @dginev indicated.

As for the "subjonctif" case, one can speak the same expression different ways in English such as a passive vs active tense: "A is a subset of B" and "A subsets B". That's a little clunky in this example, but not at all with an operator like =: "A is equal to B" and "A equals B". In the example in you have in French, is using the "subjonctif" variation a stylistic choice or is only one form proper French?

[polx]

In your example, it seems that knowing that A is a set of female mathematician results in different speech than if you know A is a set of male mathematicians. Is that correct?

Nope. A surface is a 2d-manifold as differential geometry teaches it. And this thing, in French, as anything, has a gender. Thus it needs to be "accord"ed with in all adjectives that qualify it. Nothing stylistic, the lack of it is just wrong. My worry is more that French has (just) two genders and that other such variations are much richer in other languages. We can't pre-encode "gender" as an attribute of the class of an isa... it will keep popping up with more genders!

In the example in you have in French, is using the "subjonctif" variation a stylistic choice or is only one form proper French?

It is a requirement of the sentence. Nothing optional stylistic. A different behaviour is just wrong French. But it might be that read-out-loud-tools should be acceptable to do that...

That can be hard to deal with and I suspect that several systems, such as the one Murray implemented in MS products, just punt on that.

@MurraySargent should probably know.

[MurraySargent]

I don’t think Microsoft's French math speech observes gender. Perhaps this isn't super crucial since AT math speech generally wants to be concise, not elegant. Sort of pseudo speech. You'd say "a sub 2", not "a subscript 2", etc. But a verbosity setting could change the degree of brevity.

[dginev]

To reply to Paul's original example, and maybe get a little headway for the issue:

Examples in French:

* A ⊂ B: (A is included in B): _A est inclus dans B_. However:

  * if you know that A is a surface then _A est incluse dans B_ (A is feminine).
  * If you write X, Y ⊂ B: then it should be pronnounced _X Y sont inclus dans B_ or _X Y sont incluses dans B_.

Luckily inclusion seems to have a differently spoken, but identically written main concept name in French: Inclusion (mathématiques)

So the intent markup for the two examples ought to be inclusion(A,B) and inclusion(_(X,Y),B) in each case (or a reference-rich variant that resolves to that).

If I knew a bit more French I could come up with some other annotation than underscore for X Y, but alas - apologies. Luckily that's not the crux of the question. I think the main concept name should keep its encyclopedic gender (as seen in encyclopedic resources for a specific language).

* with just the same symbol (it could be most others): the "subjonctif" variation may be needed: _Afin que A ⊂ B_ (So that A ⊂ B) should be pronounced _Afin que A soit inclus dans B_ (est has become soit)

Sounds like a case of recognizing context for speech synthesis, maybe even one where a neural model would do a better job than a symbolic AT :> Luckily we have some recent tools which do not require annotating the full linguistic details to synthesize an eloquent speech string.

I currently like the simplest approach - one intent entry per encyclopedic concept, and leaving the linguistic complexities to smart AT implementations (or to pedantic annotators aware of the _literal overrides).