stuck with functional speech?

[NSoiffer]

This issue pulls the essence of the question I raised about chemistry in #397 and which is also part of prefix/infix/postfix speech mentioned in #256. This issue only applies to non-core intents. In #397, the proposed solution is to move the chemistry notations into core. This doesn't solve the underlying problem I was trying to point out in that issue.

On the call today, I think it was @davidfarmer who brought up another example. It may have been p-norms or something else. I'm going to stick with p-norms as an example because they point out a problem. The notation is $||\mathbf{x}||_p$ with a MathML encoding:

<msub>
   <mi mathvariant='bold'>x</mi>
   <mi>p</mi>
</sub>

A very reasonable intent value would be intent='p-norm($base, $p)' where the p and the x in the above example are marked appropriately. Being in "open", this would likely get a verbal rendering something like "p norm of x and p". Not ideal, so maybe one would write intent='p-norm($base)' to get "p norm of x".

But what if the subscript is 4? I think in that case you want to hear "4 norm of x". Maybe that's not the standard speech for this construct, but for another notation, it might be desired if it isn't here. So for the sake of discussion, assume that it is the desired form to pronounce this notation. That means the subscript should either be one of the arguments or be part of the name of the function. If the subscript needs to part of the name of the function, that means that one can't put into the open list "p-norm" and be done with it, one needs to say that there are an infinite number of names 2-norm, 3-norm, etc. I don't think that notion currently exists in the open list. Note that those names (2-norm, etc.) aren't actually valid with the current syntax since you can't start a name with a digit, so two-norm, three-norm, etc.

Clearly there is a parameter here and the problem is that although intent is meant for speech, intent forces a single style of speech for notations in open. I don't want to make intent have a complicated syntax, but I also want intent to be able convey what the author wants to be said for a given notation. In this case, it is a more infix style. This could be conveyed as silent-intent="p-norm($p, "norm-of", $x)". Another option could be infix-intent="norm-of($p, $x)" where the function head is intended to be used between the args. A silent-intent would have solved the chemistry speech issue in #397. In #256, I also proposed prefix-intent and postfix-intent, but with silent-intent, there is no need for them.

To reiterate what I think is the goal of intent: to provide enough information to AT so that the notation can be spoken in a manner consistent with the author's intent. Semantics are a nice plus, but not a direct goal. The current solution of a single intent allows the author to distinguish between two notations, but their control over how to speak it is limited to a functional style (or handled by AT for core names) so I feel that we have only partially solved the problem we set out to solve.

[davidcarlisle]

@NSoiffer I don't like variant attribute names as for example infix-intent works for a single call, but nested expressions harder to manage. to square the norm you might want intent of power(norm($p,$x),2) but you have no way to give infix reading of the subterm.

If we want to do this I think we could simply allow infix intents, I would not allow parens except the existing use for function application, or allow operator precedence but you could allow left to right interpretation of infix binary operators.

intent      := name | number | reference | application | infix
name        := NCName
number      := '-'? digit+ ( '.' digit+ )?
reference   := '$' NCName
application := intent '(' arguments? ')'
infix       := intent intent intent 
arguments   := intent ( ',' intent )*

would allow intent="$p norm-of $x" and intent="square($p norm-of $x)"

As with prefix function you could restrict the middle term (eg just NCName) but here I used intent so it also allows $x $y $z and 1 2 3

[dginev]

I still like my initial suggestion about addressing fixity, so I'll present it again briefly.

A dedicated column in the global "list of names" can provide a directive for the produced narration. Currently that column is named "Speech Hints", but that name is tentative.

The same intent value could be used in different fixities, which leads to different narration. A basic example from the spreadsheet is minus, which has the two speech hints $1 minus $2 and negative $1, for infix/prefix use respectively.

A more advanced example from the Open list is moment-of-function, which can be spoken as $1-th moment in a single-argument notation, or $2-th moment of $1 when used in a two-argument notation. (Note that the referenced digits $1,$2 that I am including here only make sense if you are looking at the "Known notation" column in the list. The list itself is meant to be self-contained.)

The details on the exact syntax in these columns can be developed further - all tentative.

The main reason I like the single global list as a carrier of this feature is that it incentivizes adopters to register the intent names they use - hence growing the list. By doing that we get to provide global benefit to all adopters by annotating just once - rather than require annotation on each use, in each expression, which is a limitation of what a MathML attribute can accomplish.

[davidcarlisle]

@dginev yes I'm conflicted really. In principle I agree with you that having a central list is better than having to annotate each expression, just as having the operator dictionary is better than having lspace=.. rspace=.. on every <mo>+</mo>

But... I am not sure I have confidence that the lists will be dynamic enough (hard to know at this stage). If somone finds they need a new intent value while marking up a document, are they really going to update the global open list?, and if they do is the screen reader the human reader of the document uses going to consult the updated list in real time? Perhaps that will happen, but I have my doubts.

Just as with <mo> space attributes, a central dictionary is more elegant but inline markup is simpler to manage in many circumstances.

So I'm very supportive of columns in the list giving speech hints for infix readings, but we may still want some inline markup that adjusts the default reading of unlisted terms.

[dginev]

@davidcarlisle well, I am tempted mention that in Presentation MathML you now have both global and local mechanisms available, at least in the HTML context.

As far as I understand MathML Core, token elements, including <mo>, can carry a style attribute that can override a very wide range of rendering specifics, including the ones in the operator dictionary. And then one can even abstract away from that using class, as is usual in HTML.

I think we would be wise to mirror such a setup, where we have an incentive in place to maintain a high quality global baseline, while also allowing some override mechanism for exceptional cases.

However, starting with all of that machinery on day 1 is probably not possible, because we don't have the resources to build it all. If we also don't have group alignment, it is easier to imagine a scenario when none of the proposals work in an ergonomic and thought-out fashion, rather than a scenario where all of them do.

[davidcarlisle]

@dginev yes I was just editing my previous comment to clarify that I was in favour of having the information in the central lists. The css example that you give is similar to the operator dictionary example. In both cases it's better to use a central definition, but you can use inline markup when needed.

[NSoiffer]

Apologies for not joining the discussion earlier, but I decided if I was ever going to get chemical expression/formula inference working in MathCAT, I need to put my head to the grindstone and avoid distractions. After several attempts, I feel I've got that under control (but not finished), so back to this discussion...

I think the operator dictionary analogy is apt, but the major difference is the operator dictionary only changes when the version changes and isn't consulted live. In fact, I'm sure every MathML "implementation" (even those that just generate MathML) out there preprocesses it to turn it into some form they can use in their program. Consulting a live table has many problems:

1. Connectivity: even in this day and age, usable/fast access to the internet is not a given. I don't believe MathCAT in NVDA is viable if an internet connection is required
2. Reliability: given that users can edit the table, that means the table can be messed up. I should know since I was someone who inadvertently messed it up by sorting on a column. If the "great masses" actually start to use it, at least in the form that now exists as google spreadsheet, the likelihood that the table gets broken seems far from small. Production code can't depend on external resources like this.
3. Repeatability: the might be a pro or a con, depending on your point of view. In general though, users expect the same result from the same input from day to day unless the user takes some action (such as upgrading the software) to cause a change. If someone can change an entry at some random point in time, this can be a problem.

I am not implying that a user-contributed table is a bad thing. My point is that reliable software will need to take snapshots of it, run it through whatever verification/testing procedures it has, and potentially transform it to a format that is more convenient/efficient for it.

Both of you have said it is useful for MathML authors to have the ability to override the entries in the table. That lower level is what I think we should focus on because that is what the MathML spec needs to say. A separate spec (or more likely "note") should describe the spreadsheet (or whatever final form we agree on).

Turning my attention back to intent...

The current form for intent is simple and so has a good chance of being used and implemented. I am very concerned that having lots of options/ways of overriding intent will destroy the simplicity of it. When I look at the speech hints column, my reaction is "why don't we just use that for intent"? For core, it is too limiting to specify it because of all the special cases (speak as an ordinal fraction if the numerator is numerator is less than 10 and ..., etc). It would be extremely complicated specifying all of those. But for open, why don't we allow intent =" $i th moment" if there is a single arg and intent = "$i th moment of $x" for the two arg example, where arg=... on the appropriate children elements?

There are parts of what I just wrote I'm not keen on:

• free text seems a bit loose to me, but tied together with - as in th-moment makes be a little less worried, Quoting (ugh) is also an option.
• no forced pronunciation -- things like "th" tend to sound pretty bad. Allowing for SSML seems like too much complication.
• this is even more English-centric than our current scheme (which I think will likely get flak from the i18n group).

On the other hand, the intent grammar gets even simpler if that is what we turn intent into.

I'm not suggesting we get into a discussion major changes to the grammar (yet). What I do feel is that we should return to the rationale for intent. Having started to try and implement it, I am finding what we have now deficient. Maybe I'm forgetting something we already discussed, but I don't see why @dginev's suggestion of having both a name and a speech hint is a good idea. @dginev: hopefully you can remind me why we need both.

[davidcarlisle]

@NSoiffer  yes I don't think we have any proposals for an API for implementations to read the intent lists in real time.

Presumably we will say implementations SHOULD support the core list, so document authors can have reasonable expectation that those terms get used as described. However, adding to the open list is just a request for implementers to add support for that term at some point, so can't be a mechanism for influencing the reading of a particular expression, we need an inline way to influence the default reading for unlisted and open-listed terms.

We could give up on the grammar altogether and instead describe it as a free text "template" which is an arbitrary string except $arg references are substituted (essentially the syntax of the current column but with named rather than numeric references). That would work well for speech hints but might mean we need to drop the idea of semi-formal uses of intent eg categorising alignment rows.

The grammar suggested above would cover at least a reasonable number of the listed speech hints although arguably abusing the functional interpretation in some cases, but picking a few examples:

$1 over $2
$1 divided-by $2
the-ratio-of ($1 and $2)

It can't really do suffix though, so $1 et cetera or even $1 factorial (if that wasn't in core) are problematic.

Of course you can use intent just on the suffix, so <mn>3</mn><mo intent="factorial">!</mo> but sometimes you want to handle this on an ancestor and reference the arguments.

Since we don't use parens for grouping, only for function calls, we could allow suffix function call ($1) factorial

So

intent             := name | number | reference | prefix-application | infix-application |suffix-application
name               := NCName
number             := '-'? digit+ ( '.' digit+ )?
reference          := '$' NCName
prefix-application := intent '(' arguments? ')'
infix-application  := intent intent intent 
suffix-application :=  '(' arguments? ')' intent
arguments          := intent ( ',' intent )*

I think that allows most of the speech hints but perhaps the interpretation of () is a little weird

$1 for $2 going-from $3 to $4 parses as  for($1,going-from($2,to($3,$4)))

But $1 for ($2 going-from ($3 to $4)) doesn't parse

$a and $b or $c is  and($a,or($b,$c))

but ($a and $b) or $c  is an error

Mathematica finesses this by using different brackets, we could for example use () for functions and {} for grouping so

{$a and $b} or $c and $a and {$b or $c} would be allowed

intent             := name | number | reference | prefix-application | infix-application | suffix-application | group
name               := NCName
number             := '-'? digit+ ( '.' digit+ )?
reference          := '$' NCName
prefix-application := intent '(' arguments? ')'
infix-application  := intent intent intent 
suffix-application :=  '(' arguments? ')' intent
arguments          := intent ( ',' intent )*
group              : = '{' intent '}'

[dginev]

@dginev's suggestion of having both a name and a speech hint is a good idea. @dginev: hopefully you can remind me why we need both.

Symbolic expressions require symbols, hence the intent names. A motivation the group had when I joined was to (provide the ability to) base a variety of algorithms for a11y on symbolic forms.

If you have a special narration rule that wants to say "$x squared" for intent="power($x,2)", and "$x cubed" for intent="power($x,3)" it is harder to know to use that rule if an author wrote intent="the second power of $x" vs intent="$x raised to the third power".

Literal strings have a certain "opaqueness" to them, where we can mostly expect AT to use them verbatim and pass them on to the speech engine. Since that was possible already at the top level via alttext, and possible on any node via aria-label, and still deemed not sufficient to build good accessible applications, the intent attribute design began as part of the community group back in 2020. This all had transpired before I joined, so I can't say much more.

Another symbolic algorithm could be to internationalize "by table". Again you have a special rule for narrating the second power in Bulgarian: "$x на квадрат" which is easier to generate from power($x,2) than the free-form English "the second power of $x".

In a sentence: the motivation for having symbols (i.e. names) was symbolic algorithms.

In 2020, that was a premise. To somehow provide an additional guidance for narration on top of an unknown symbol, we need a "speech hint" of some sort attached to it "somewhere". This was mostly motivated by my prototyping of an Intent Open list.

[davidcarlisle]

In 2020, that was a premise. To somehow provide an additional guidance for narration on top of an unknown symbol, we need a "speech hint" of some sort attached to it "somewhere". This was mostly motivated by my prototyping of an Intent Open list.

Perhaps, but perhaps if we relax the syntax a bit, we can use the default "read the intent names" . So still allowing the benefits of using "known" intent values, that they can have different readings as specified in the intent lists, but not forcing a prefix functional reading for unkown terms.

[dginev]

not forcing a prefix functional reading for unkown terms.

@davidcarlisle That much is not forced, unless we demand that it is.

The intent spec is already quite complex, thanks to having references to the extent that we do. A complexity which I worry has not been fully explored (or appreciated) by group members. There is an implied capability of using references alongside with the presentation tree to derive better narration.

Namely:

<mrow intent="$op($arg)">
  <mi arg="arg">x</mi>
  <mo arg="op" intent="semifactorial">!!</mo>
</mrow>

An AT implementation could look into the presentation tree and check adjacency between the referenced arguments, as to determine the order in which they could be spoken. My tiny prototype implemented that naively about 2 years ago - you can check the "factorial" example, where the narration is put together by doing exactly such a check over the presentation layout of the referenced "arg" nodes.

However, this is only possible with fine-grained annotation, that maximizes the use of references. That may not always be preferred (and may require additional mrow scaffolding). But it happens to also be the annotation style that maximizes the quality of expression navigation. Also yes - navigation - that was the third symbolic algorithm that was a motivation back in 2020.

It is very easy to spiral into having a very complex and extensive set of capabilities in the intent syntax. That continues to concern me.

[brucemiller]

I think I'm getting a bit confused about the mix of ideas (maybe I'm not alone?): @NSoiffer mentioned templates like words words $1 words where the expanded intent for whatever $1 is would be inserted. @davidcarlisle proposed some syntax modifications which I was reading in that context. Pure templates would be simple and don't require any particular intent grammar, but they inhibit i18n and special rules (eg. for power). Mixing templates into a symbolic grammar requires annoying delimiters.

But now I think that @davidcarlisle new syntax was an enhancement trying to avoid introducing templates? Alas, it introduces non-trivial parsing into the intent grammar. It also kinda obfuscates the potential distinction of whether a mathematical notation is infix/prefix/postfix and whether (in any particular language or context) it's speech would be.

If it's mainly infix/prefix/postfix distinctions that are the concern, what about an explicit speech hint, as @dginev hinted (:>). So, picking a random unused character and a weird example, we might introduce a suffix like

going-from@infix($a,to@infix($b,$c))

This keeps the grammar relatively simple. My inclination would be to allow that modifier on any intent in the grammar, rather than just ncname or references. The point is that it would be a speech hint which the AT is free to use or ignore as it wishes.

[davidcarlisle]

An AT implementation could look into the presentation tree and check adjacency between the referenced arguments, as to determine the order in which they could be spoken.

Yes it could although (perhaps) that prevents the other possible desired use of over riding the visual order by giving an intent with a different reading. (Although maybe that isn't a major concern)

But actually yes you have a point.

My grammar experiments above were seeing far we could go to allow freedom of order in the expression (as I don't really like the idea of just a free text template just instantiating references)  I think technically something along those lines could be made to work but perhaps it puts too much strain on the grammar (or on people trying to use it).

Perhaps we should merge PR #394 then call it done, and then explicitly try to mark up any problematic examples, as in your semifactorial example above.  If we can handle all the cases we have, then we can write some more words in chapter 5 and finally have something to publish for a first release.

@brucemiller 's hint extension also looks a simple possibility so that would be

intent       := name | number | reference | application 
name         := NCName
number       := '-'? digit+ ( '.' digit+ )?
reference    := '$' NCName
application  := intent hint? '(' arguments? ')'
arguments    := intent ( ',' intent )*
hint         :=  '@' ('prefix' | 'infix' | 'postfix'  | 'silent')

[davidcarlisle]

with the above, I think @NSoiffer 's example at the start of this issue would be

<msub intent="norm@infix($p,$x)>
   <mi arg="x" mathvariant='bold'>x</mi>
   <mn arg="p">5</m>n
</sub>

where the intent list gave an infix reading for norm as $1-norm of $2 so 5-norm of x or if norm was unknown, a default infix reading would be 5 norm x which isn't terrible and a document author could force a reading (at the expense of some other interpretations) with intent="norm-of@infix($p,$x) which would have a default reading of 5 norm of x

[dginev]

FWIW, the way I'd use such a syntax for this example is

<msub intent="$p@prefix(norm)($x)">
   <mi arg="x" mathvariant='bold'>x</mi>
   <mn arg="p">5</m>n
</sub>

and expect the narration $p norm of $x.

Then again, maybe it is really

<msub intent="norm@postfix($p)($x)">
   <mi arg="x" mathvariant='bold'>x</mi>
   <mn arg="p">5</m>n
</sub>

Actually yes, all 3 of the above. Which is to say - none of the above.

There is something very honest about calling a literal/text piece, a literal, textual piece. So intent='text("$p norm")($x)' fits better in my mind, if we must.

[brucemiller]

Huh, whut? Are you trying to confuse me? I'm glad that you also don't know what ($p@prefix(norm))($x) should generate. But assuming only the last bit is what you're suggesting. Which isn't in fact literal, nor honest, since it is a template. :>

Introducing a keyword (or syntax) for literals provides some capabilities, but would seem to complicate implementation, especially with templating.

[dginev]

Huh, whut? Are you trying to confuse me?

Yes! I am trying to relay how confusing extra syntax gets (and how quickly), so your reply makes me feel understood.

Which isn't in fact literal, nor honest, since it is a template. :>

That's true on the letter of what I said, indeed. A pure "honest" text annotation would have been text("five norm").

The textual templates (also called "string interpolation" in programming languages), are a feature on their own merit that requires a special implementation bit. And my original "speech hint" column certainly relied on string interpolation, without saying so explicitly.

Another in-between thing is a concatenation keyword (has been discussed in the past), that stays in the symbolic realm and avoids literals. As in a lispy cons($p,norm)($x), or cons($x,norm,of, $p), etc. Implemented as "narrate arguments in sequence without any added connectives". Usual caveat - cons or some other keyword to the same effect - doesn't really matter.

Edit: missed the explicit of!

[brucemiller]

I'm not too fond of either literals or templates, but could go along if it solves a need. @NSoiffer should chime in.

To the extent we need literals, that could be handled either by introducing some quoting delimiter (which would exclude it from containing quotes), or a keyword (eg. "text", or whatever you like) and give the text as the argument (which would exclude ")"). And if the literal could serve as a template with interpolation, as you say.

But all this adds implementation complications as well as user confusion, so only if it really solves a need.

[davidcarlisle]

One reason I keep providing complete explicit grammars is it  helps to clarify what is being suggested, even if that mostly just clarifies the suggestions are not popular. Currently we don't allow grouping parentheses, only around function calls.  so ($p@prefix(norm)) isn't currently allowed (well neither is @ but easier to guess where to extend for that. adding grouping parens is possible of course but you have to have some rules somwhere to avoid the ambiguity with function calls.

cons($p,norm)($x) is another viable possibility, which you can also think of as silent($p,norm)($x) Ie just read the arguments $p and norm without reading the head). This works well here where you can construct a compound function head so p-norm, which is then still a prefix function. Possibly less well for an arbitrary infix function cons($x,wibble,$y) maybe it's OK but the wibble@infix($x,$y) suggestion does preserve some aspects of recording the functional nature of the expression with wibble being an infix function of two arguments

[davidcarlisle]

A version with literal strings as discussed on the call just now:

intent       := name | number | literal | reference | application 
name         := NCName
number       := '-'? digit+ ( '.' digit+ )?
literal      := '"'  [^"]* '"'
reference    := '$' NCName
application  := intent hint? '(' arguments? ')'
arguments    := intent ( ',' intent )*
hint         :=  '@' ('prefix' | 'infix' | 'postfix'  | 'silent')

allowing

"Joe blog's function" ($a, 5, "p-norm of something")

as equivalent to

Joe-blog-s-function ($a, 5, p-norm-of-something)

except more explicitly non-dictionary terms, and more natural handling of non NCName characters

[davidcarlisle]

I don't like using quotes as above as I think it makes it look too much like a string value rather than an unparsed-intent.

an intent of sqrt(9) is 3, with 9 denoting the number 9 but (I think) in an intent length("vector of 1, 2 and 3") the implied expression has value 3 (the length of the vector) not 20 (the length of the string).

I'm not convinced we need these, or we could use a different character such as back quote:

length(`vector of 1, 2 and 3`)

[brucemiller]

Quoted literals to refer to Joe's blog may have limited usefulness, since they do need to appear within a quoted attribute value. If we really want literals, we may need to find some other characters to abuse. Jus' sayin'

[davidcarlisle]

Quoted literals to refer to Joe's blog may have limited usefulness, since they do need to appear within a quoted attribute value. If we really want literals, we may need to find some other characters to abuse. Jus' sayin'

I'm not keen on having them at all, or we could use backticks, if we use normal quotes as above, it doesn't allow " in the string. Xpath 1 allowed " delimited and ' delimted strings so you could have either quote (but not both). xpath 2 allows a doubled outer quote so "ab'c ""xyz""" is the string ab'c "xyz" with both quotes, of course to put the xpath in an xslt attribute you need (as would intent) xml escape as well so select='"ab'c ""xyz"""'

[brucemiller]

It seems that we've come up with several ideas:

• quoted literals
• interpolated quoted literals
• speech hints
• intent grammar extended to infix, postfix Certainly not all of these are needed to solve the perceived problems --- assuming any of them do. And certainly each of these is strongly disliked by someone.

It would be good to assess what the minimal set(s) of these features would be needed.

[dginev]

Bruce has found a path through my objection of using Intent Open for "literal text" in our post-meeting today, which we can discuss tomorrow.

As long as we make a clear conceptual separation from "the Open list" and "everything else in that syntax", I think it is palatable to reuse the dashed keywords also for the literals.

An example of what I mean by a "distinction":

• rename the Open list (aka "Level 3") to "Intent Open Names"
• refer to the open-ended syntax of dashed-NCName-pieces-of-any-variety as "Intent Open"
• maybe rename the name grammar rule to name_or_literal := NCName and remove the new literal rule.

With a change of this flavor (likely this isn't the best concrete setup), we should be able to avoid quoted literals in the intent grammar entirely and make due with Joe-blog-s-function($a, 5, p-norm-of-something)

[davidcarlisle]

@dginev sounds promising, certainly I'd like to avoid adding strings.

Personally I don't see any logical distinction between the open list and unlisted, but if a mild difference in terminoligy as you suggest gets us out of an impasse I'm not going to object. The difference is always going to be less than "clear conceptual separation" because if I use carlisle-function($x) as an unlisted intent and the name catches on, someone may add it to the open list, or perhaps it proves really fundamental and it gets added to core. The document instance and the intended interpretation haven't changed, just the listing status of the symbol. I would expect that listing used, but previously unlisted, terms would be the normal way things get added to the open list.

[dginev]

@davidcarlisle luckily, there is a difference, your example simply fits in the "Name" class and is welcome to the list. Examples that will never be welcome to the open list are:

• the-function-defined-on-page-3
• carlisle-s-function-before-2022
• square-root-of-four

As long as I can advertise a design where the Open Names list is aiming to collect "encyclopedic names", there will be pronounceable literals that clearly do not belong on it.

[davidcarlisle]

@dginev sure and it's OK to hope that's how it turns out, but that's the thing about open lists, we don't get to choose. If I want to add i-think-this-function-exists-but-i-havent-proved-it-yet what happens? My assumption about an open list is that it gets added, modulo some basic, perhaps automatic, moderation to prevent spam and abusive texts. Maybe the open list isn't as open as that and we need ongoing moderation, or more wikipedia-style: items can be added but if not deemed useful they can be deleted. I can see it going either way, I just wonder how you see such an entry being handled

[dginev]

If I want to add i-think-this-function-exists-but-i-havent-proved-it-yet what happens?

I may commit to moderating the list part-time, ideally recruiting other moderators. If we move to a PR-based workflow, contributions will be easier to track than the google sheet, but even in the current sheet one can check the history and edit down bad contributions.

The "open" nature of the list is that contributions are welcome at any time, and any concept with some documented precedent gets accepted, not that it will never get any quality control.

[davidfarmer]

There are some common situations where it might make sense to have specific markup:

1) Name of a function (eg, the Ackerman function)

In this case there could be markup specifying that something is a function, along with the name of the function.

2) Infix operator (eg, the "free product of A and B")

There are two inputs, and an operator which sits between. Intent indicates that is the setup, with a parameter to specify the name of the operation.

3) Decoration, such as a tilde on top of a letter, (eg, the "double cover of M")

Could be prime, tilde, star, dagger, etc. Where the decoration goes does not matter to the intent -- only that the intent indicates a decoration. A parameter says how to say the decoration.

If we define appropriate ways to add intent for these and a few other common cases, that would decrease the creativity required of the author. And be better for internalization.

There would be special entries in the official table indicating how to use these types of intent.

[davidcarlisle]

If I want to add i-think-this-function-exists-but-i-havent-proved-it-yet what happens?

I may commit to moderating the list part-time, ideally recruiting other moderators. If we move to a PR-based workflow, contributions will be easier to track than the google sheet, but even in the current sheet one can check the history and edit down bad contributions.

The "open" nature of the list is that contributions are welcome at any time, and any concept with some documented precedent gets accepted, not that it will never get any quality control.

Hmm we could do that, which is basically the model OpenMath uses for "Contributed" as opposed to "Official" Content Dictionaries, I had assumed that "Open" here meant something more lightweight with essentially wikipedia-like or stackexchange-like access with essentially open write access, but with editorial control required to take down offensive content, block DOS attacks etc.

If we moderate entries on mathematical purity rather than legal text grounds, that may be a good thing but means the mechanisms are basically the same for open and core (and "open" is perhaps not a good name)

[davidcarlisle]

@davidfarmer If I understand your examples they seem to be normal cases where the decoration is in the presentation tree and the intent sits over it, eg

free product of A and B

<mrow intent="$op($a,$b)">
<mi arg="a">A</mi>
<mo arg="op" intent="free-product">*</mo><!-- or whatever symbol-->
<mi arg="b">B</mi>
</mrow>

double cover of M

<mover intent="$op($x)">
<mi arg="x">M</mi>
<mo arg="op" intent="double-cover">~</mo>
</mover>

[davidcarlisle]

@dginev trying to follow

https://github.com/w3c/mathml/issues/398#issuecomment-1178334905

OK something like

intent          := name-or-literal | number | reference | application 
name-or-literal := NCName
number          := '-'? digit+ ( '.' digit+ )?
literal         := '"'  [^"]* '"'
reference       := '$' NCName
application     := intent hint? '(' arguments? ')'
arguments       := intent ( ',' intent )*
hint            :=  '@' ('prefix' | 'infix' | 'postfix'  | 'silent')

Or without PR 394, application replaced by

application      := function '(' arguments? ')'
function         := name-or-literal | reference | application

With associated text along the lines of....

Here name-or-literal MAY refer to an entry in the list of Intent Concepts supported by the application.

• In this case the entry may give (possibly locale-specific) speech hints or other specific customisation of the intent handling.

  An application SHOULD support the concept names in the core list and MAY support names from the Open Intent list

  If the Name is followed by a hint in a function call, the application may use the hint to choose a suitable reading.

• If the name-or-literal does not reference a Concept Name listed by the application, a default reading should be used, replacing [-_.] by space and then reading the name as given.

  In a function call, if the name is followed by a hint the reading of the name may be suppressed, or precede or follow the reading of the arguments, or may be repeated between each argument.

So:

• element-of($x,$set) which is in the core list may be read as ... is in .. (or other readings see ... core list url)

• the-function-defined-earlier($x) would be read as "the function defined earlier of ..."

• mydiv($x,$y) would be read as mydiv of .. and ..

  mydiv @infix ($x,$y) would be read as .. mydiv ..

  mydiv @silent ($x,$y) would be read as .. ..

Contributions to the Open Intent list are welcome. See ... some URL for submission instructions.

[dginev]

@davidcarlisle Thanks, I appreciate the unrolled summary. I am still holding off on the "hints" idea, until it becomes completely clear what class of narration problems we are tackling.

@brucemiller had asked me to come up with some concrete grammar alternatives based on my thinking, to get an idea of where we are diverging in our design directions. So, with literals allowed in Intent Open, my mental model is:

intent          := name-or-literal | number | reference | application 
name-or-literal := NCName
number          := '-'? digit+ ( '.' digit+ )?
reference       := '$' NCName
application     := function '(' arguments? ')'
function        := name-or-literal | reference | application
arguments       := intent ( ',' intent )*

The next piece I would personally like to resolve is what to do with the dual question of "silence" and "quiet lists". Without @hint or the literal empty string "" we (once again) lose the ability to mark a node as silent.

I think I like the idea of using the empty intent value for a silenced node (intent=""). That could be added as:

intent          := name-or-literal | number | reference | silent | application 
silent          := ''

The "silent adjacency" feature is a bit more slippery. I tried adding a separate "list" to the grammar, but the grammar grows quite a lot, and gets tricky to reason about, so I think that's not the right approach. Adding custom syntax also adds complexity, so I myself wouldn't volunteer that either.

We do have a list already available through the arguments rule. And suddenly I see an interesting opportunity for a "middle path". If I now agree to using intent as the function head rule, and distill:

intent          := name-or-literal | number | reference | silent | application 
silent          := ''
application     := intent '(' arguments? ')'

Then it becomes legal to write intent="(foo,bar)" or intent="($p,norm,of,$x)", which can have the special convention of "concatenate narration with no connectives" on the basis of the empty string being the "silent" intent. While this is internally consistent, I still wonder if there isn't something even more orderly of a setup that is even clearer to communicate to adopters.

This may be an approach that saves us from adding another sigil (@), also without introducing new positional rules (intent intent intent). So at the least it seems like a useful direction to think in.

[davidcarlisle]

@dginev I added some words/examples for hint to my comment above so the description matches the grammar.

Compared with empty intent proposal in https://github.com/w3c/mathml/issues/398#issuecomment-1178941136

I suspect they both cover the same use cases for silent reading

Not adding @ makes the grammar simpler and the resulting expressions look nicer

But you get silent reading of the function at the cost of dropping information that the function was there,

Conversely adding @ especially if documented as a "hint" rather than an "instruction" is less direct control over the reading but still maintains some possibility of using the intent as a hint to interpreting the MathML for computation or other uses. Of course semantics was the original name here, and while we have agreed where the design aims conflict to favour accessible readings over computation, I'm not sure we have to conflict here. (a,b) looks nicer than foo@silent(a,b) but only gives the information that a and b are combined, not by what,

molecule@silent(atom-count@silent(H,2),O)   looks worse than ((H,2),O)   or even
(H,2,O)
and all three may be read as H-2-O, but the first seems truer to the aim of intent to enable better readings by giving structural information (rather than by supplying a reading)

[dginev]

the first seems truer to the aim of intent to enable better readings by giving structural information (rather than by supplying a reading)

That purpose is met without the hints, as in molecule(atom-count(H,2),O).

The "silent" treatment is just one of many possible overrides one may need for narration. As you've identified, fixity is another. Connector words are a third. Let's pick group algebra since it has two possible narrations, and we can imagine an author wanting to override one for the other. The notation R[G], could be:

<mrow> <!-- the mrow scaffold is optional -->
 <mi>R</mi>
 <mo>[</mo>
 <mi>G</mi>
 <mo>]</mo>
</mrow>

Expected narrations here are group R-algebra as well as group algebra of G over R. I will focus on the second narration for now, and the key task is to somehow introduce the "over" connector word.

If this was in Intent Core, a simple group-algebra($R,$G) may have sufficed, with AT taking care of the details. But this is in Open, so we need to 1) add "group-algebra" and relevant "speech hints" to the global list and/or 2) provide "narration overrides" in every MathML expression that needs them.

With @hint, I could imagine all of:

• intent="group-algebra(over@infix(G,R))">
• intent="over@infix(group-algebra(G),R)">
• intent="words@silent(group-algebra,of,G,over,R)"
• intent="group-algebra-of@prefix(over@infix(G,R))"

and various others. I'm hoping to illustrate that as soon as we start dealing with connector words, the symbolic language becomes more of an artifact than anything concretely "semantic".

With the "silent" idea, I can imagine:

• intent="group-algebra((G,over,R))"
• intent="(group-algebra,of,G,over,R)"
• intent="(group-algebra(G),over,R)"

To reiterate for clarity, the target for all of these is the narration

"group algebra of G over R"

which an author is trying to make an AT system produce on the same day, without any updates to global lists or other developer intervention.

[brucemiller]

I'm comparing @dginev's previous examples with an eye to whether they preserve any "semantics" or not. Not that I necessarily believe that is the most important issue.

|molecule(atom-count(H,2),O)|.

With no hints, and no open dictionary, this would read quite clumsily, but be somewhat semantic.

• intent="group-algebra(over@infix(G,R))"
• intent="over@infix(group-algebra(G),R)"
• intent="words@silent(group-algebra,of,G,over,R)"
• intent="group-algebra-of@prefix(over@infix(G,R))"

The 3rd example seems to have lost all semantics. Assuming that "group-algebra" and "over" were added to the open dictionary, in a way suggesting the expected arguments, then only 1 of the remaining 3 would likely "conform" to those semantics. Even without the open dictionary, those 3 are sorta semantic.

All 4 of these could give an appropriate speech, even without the Open dictionary, however.

• |intent="group-algebra((G,over,R))"|
• |intent="(group-algebra,of,G,over,R)"|
• |intent="(group-algebra(G),over,R)"|

These may provide the desired speech, but none of these do much to preserve semantics, however. (imho)

To summarize: it would appear that speech hints might allow you flexibility to get the desired speech patterns, without needing to lookup entries in the Open dictionary. And at the same time tends to better preserve semantics, which is even further enhanced for applications that can & do consult that dictionary.

So you've made speech hints look more helpful, although they admittedly look awkward to type.

[davidcarlisle]

and various others. I'm hoping to illustrate that as soon as we start dealing with connector words, the symbolic language becomes more of an artifact than anything concretely "semantic".

Sure but I don't think that is an issue really. If we wanted to give the author full control over the reading, we'd supply full text strings. hint is in the spirit of the rest of intent which is to annotate the presentation to allow AT to come up with better readings than it would make from just the presentation tree, not to provide a full control over the reading.

• intent="group-algebra(over@infix(G,R))"
• intent="over@infix(group-algebra(G),R)"
• intent="words@silent(group-algebra,of,G,over,R)"
• intent="group-algebra-of@prefix(over@infix(G,R))"

Yes an author could do any of these, personally I'd recommed that they don't, I'd rather they use group-algebra(G,R) and accept the reading generated (or for a specific AT system get group-algebra added to its dictionary, possibly via the WG Open list.)

@ is useful for

mydiv@infix(a,b) read as a mydiv b

myfactorialthing @postfix (n) read as n myfactorialthing

so a simple hint making a big improvement in the default reading of many common cases. if it is inconvient and not that good for injecting english connector words like of and over into the reading then so be it. Actually your examples show that if someone really wants to force the reading they can do so, although without hints at all they could do

intent="group-algebra-of-G-over-R"

if the idea is to force the reading at the expense of dropping any semblance of marking up the structure.

So we can't stop such uses, in fact I'd say allowing them is part of the "open" design but they are not a primary use case for proposing @hint

[NSoiffer]

Just to be clear -- what follows is brainstorming, not a thought-out proposal...

"group algebra of G over R" is an interesting and I suspect not that unusual case. A thought that popped into my mind was to generalize the @ notation (hint) a little:

hint         :=  '@' ('prefix' | 'infix' | 'postfix'  | 'silent' | 'name-or-literal')

The idea is that if the hint is not one of the four listed, name-or-literal would be used as a connective. So for the example

group-algebra@over(G,R)

The basic idea is that when reading a functional notation, there is actually a default now to separate the arguments. Typically it is "comma" although sometimes we say "and" (e.g., "f of x and y"). By specifying an alternative connective, perhaps a majority of cases where "comma"/"and" are not appropriate is captured. This will not solve all cases, but I think captures a lot of them. To go further, you probably end up with yet another mini-syntax: `foo@(first, second, ...) for a connective word to be used the first time, the second, etc., with the last one repeated as necessary (akin to what is done in MathML for some table attributes). That might be too big a step at this point (but a future version could do that without conflicting with this simpler idea).

FYI: it's a very, very minor point, but I think : might look a little better than @. Compare the above to

group-algebra:over($G, $R)

or

semifactorial@postix($x)
semifactorial:postix($x)

[davidcarlisle]

@NSoiffer not keen on : as while it is not allowed in NCName foo:bar looks a lot like Name which is the form of xml elements and where foo is a namespace prefix.

Actually no character is needed and you could use space, the grammar could disambiguate two NCName before a (.

group-algebra over ($G, $R)

I'd rather not add more to this PR though, if we might go in the direction you indicate it would be simpler to merge this then have a new issue or PR on further extensions.

[NSoiffer]

My idea is just brainstorming -- it is not ready to be incorporated into text although I think I'll see how it might help in MathCAT.

[brucemiller]

Also not too fond of :. And I woulld worry that a space might lead to confusion, since "-" (in a name) sounds like a space.

[NSoiffer]

Ok, I drop my suggestion to switch from @.

[dginev]

@NSoiffer

"group algebra of G over R" is an interesting and I suspect not that unusual case.

Indeed.

free algebra is notationally similar to group algebra, but uses a different connector (on instead of over), and the narration of the symbol is interrupted by one of the arguments.

The notation can be presented as:

<mrow>
 <mi>R</mi>
 <mo>⟨</mo>
 <mi>X</mi>
 <mo>⟩</mo>
</mrow>

with one possible intended reading of free R-algebra on X. The intent concept name (following the encyclopedic rule) is free-algebra. How does one override the narration in the functional spirit of @hint? Is it:

intent="free-algebra@silent(free,$arg1,algebra,on,$arg2)"

or towards a new

intent="free-algebra@argsplit@on($arg1,$arg2)"

with an introduction of a special keyword (such as argsplit) for splitting the free and algebra from each other via the first argument? Maybe that should be arg1split, arg2split, ... to have finer grained control of which argument should separate the intent value. Or is that degree of override of narration considered as out of scope for @hint?

---

Readers curious of alternatives could compare this to a flat narration-near enumeration that loses the symbolic connection to free-algebra entirely:

intent="(free,$arg1,algebra,on,$arg2)"

Alternative syntaxes that may seem more palatable would require an addition of a preferred approach to writing down lists. For 3 quick examples, it could use comas, spaces, or a dedicated set of fences, maybe brackets:

intent="free,$arg1,algebra,on,$arg2"
intent="free $arg1 algebra on $arg2"
intent="[free $arg1 algebra on $arg2]"

At a much later date, as support of AT systems broadens, such narration overrides could be refactored to the simple symbolic form:

intent="free-algebra($arg1,$arg2)"

provided that speech hint support is present in the Open list and adopted in AT, or that a concept gets promoted to the Core list.

---

As to @davidcarlisle 's summary that

hint is in the spirit of the rest of intent which is to annotate the presentation to allow AT to come up with better readings than it would make from just the presentation tree, not to provide a full control over the reading.

We see this differently. I see no issue with providing "full control over the reading", especially now that we have decided not to interoperate with ARIA (which is a recent change).

In the case of an unknown symbol such as foobar@infix, the fixity information of @hint adds a narration-oriented positional directive, informing AT of where to put the baseline pronunciation of the unknown symbol that the @hint annotates with respect to its arguments. The (semantic, not AT) benefit is that the symbol is still explicitly mentioned. At the cost of a new DSL that potentially uses a different fixity value from the notation fixity in the Presentation tree. I find that confusing. To me it also quickly starts feeling artificial as we use it for different notational examples sampling from the long list of English prepositions. Language is messy.

The trade-off of

1. introducing a new sigil @ and inventing a new mini DSL vocabulary
2. allowing a narration-near flat enumeration of pieces

can be evaluated differently depending on the design scale one chooses to weigh it on. To me an explicit "narration override" problem should get a language-near solution, not a symbolic one.

That way any narration problem can receive a complete remediation without external AT support, while the symbolic annotations can be kept for baseline readouts and fully supported Core/Open concept names. I see clarity in that restriction.

[NSoiffer]

Going with the extended hint syntax I threw out to see if it sticks to anything (along with a small change from my original thought):

<mrow intent='free-algebra@(free, algebra-on)($r, $x)>
   <mi arg=$r>R</mi>
   <mo>⟨</mo>
   <mi arg=$x>X</mi>
   <mo>⟩</mo>
</mrow>

to get the reading free R algebra on X.

In this version, if a list is given for the hint, the first hint arg is read first, then the first element arg ($r), then the second hint arg, etc. This is a pretty general solution if an empty string is permitted. For example, although one could/should use the simpler

intent='inner-product@infix($x, $y)

to get a reading x inner product y. One could use the longer, less concise extended syntax

intent='inner-product@(,inner-product)($x, $y)

This form keeps the concept name intact, allows a general reading, and still leaves the conceptual notation for the args separated from the words. It is not the prettiest notation, but is simple for a program to use.

[davidcarlisle]

I'd agree something like intent='inner-product@(,inner-product)($x, $y) is probably workable, but I'm not convinced it's desirable. I'd rather defer it to mathml-next (or at least post first working draft). Possibly extending @hint to arbitrary NCName read as literal we could do now, but if we are considering adding (....) later, perhaps better to just stick to the keywords now?

More generally I don't see forcing exact reading, epecially exact use of connectives, as in scope for intent. Even something simple like a fraction has many possible readings, and the role of intent is to disambiguate the notation so you know it really is a division fraction so AT can generate something not confusing. It is not aimed at forcing any particular reading. I would read 1+\frac{a}{b} and \frac{1+a}{b} the same way (one plus a over b), but I can use the fraction bar to know the scope. AT readings are just different to conventional readings.

So if free-algebra($R,$G) is read as "free algebra of R and G" that is OK. Adding @on with Neil's NCName extension is an additional hint we could add. But (like p-norm example) I'd only try to force "free R-algebra" if, in a given context, the same field is used often enough that can be considered a constructor in its own right, so free-R-algebra($G). If you really want to force a templated version the current proposal allows that as free-algebra-construct@silent(free,$R,algebra-on,$G) or some such with, as usual, some loss of semantic information, and language neutrality in favour of forcing a reading.

[NSoiffer]

I think @davidcarlisle's goal to get something out and thus "keep it simple" is a good one. But it is also a good goal to solve the problem we set out to do: provide a reasonable accessible rendering for unknown notations. I also feel part of the goal is to also define the notations AT should do a good job rendering (i.e., core). So there's a tension. For now, I'm ok with leaving out the extended syntax I proposed. It is beginning to grow on me as it brings together the various goals of conceptual templates and control over speech into a single framework.

One change I think I'd make is to use brackets rather than parens as there are parens floating around for other things. For example:

<mrow intent='free-algebra@[free, algebra-on]($r, $x)>

Using braces would be another option:

<mrow intent='free-algebra@{free, algebra-on}($r, $x)>

[davidcarlisle]

@NSoiffer comparing

free-algebra@[free, algebra-on]($r, $x)

and the currently allowed

free-algebra-construct@silent (free, $r, algebra-on, $x)

I'm not sure the additonal syntax gains a lot. A slightly clearer function head but the connecting words out of sequence from the arguments so it's a bit harder to read.

[AdamSobieski]

What about uses of intent hints for expressing the desired prosody for speechified mathematics expressions?

Prosodic hints could pertain to: intonation, stress, tempo, rhythm, pause, and chunking. Intonation and stress, for example, work together to highlight important words or syllables for contrast and focus. This is sometimes referred to as the accentual function of prosody.

Using bold text as an indicator of increased voice volume or emphasis, we can envision two spoken expressions:

1. x + y
2. x + y

Could intent hints be of use for precisely describing the desired prosody for speechified mathematics expressions?

[NSoiffer]

@AdamSobieski: prosody is useful and SSML already incorporates it. But specifying that SSML (which is a W3C rec) is allowed is a big complication, especially when the speech engine (e.g., Eloquence) isn't SSML. In such a case, one would have to not only parse the SSML, they would have to translate it.

[NSoiffer]

I agree that putting the speech into the arguments is easier to read. However, it muddles speech with the conceptual template. I was trying to avoid that with my suggestion and keep the two separate, but close together. Although one can almost always ignore the non-argref arguments, it isn't always possible to get back the proper arguments. For example, we had the integral example a while back that had 1 for the numerator.