references in a function head

[NSoiffer]

Options

There seem to be differences in understanding how function heads should be processed if they are a reference. I believe there are three options that have been discussed:

1. Bottom up: substitute the referenced element into the place it is mentioned. The parent's properties are prepended to any properties on the child. If the current element is referenced by an ancestor, then continue the recursion upward
2. Top down: look at any properties on the parent, convert that into a tree (e.g., move a referenced element with property "postfix" to the last child of the tree, recurse to the children. The name of the new element is either the given intent or is essentially an mrow (something that is just a container for the children).
3. Don't allow references for function heads so the discussion is mute.

I find the last option appealing because to-date, no one has come up with a compelling case for needing to use a reference in the head. As discussed in the call today, if there is a core function named "apply", then higher order functions can be handled that way, just as they are done in content MathML.

Example 1

In the absence of a compelling case, here's one that was discussed during the call:

<math>
 <msup intent='$op:function($arg)'>
  <mi arg='arg'>x</mi>
  <mi arg='op'
      intent='transpose'
      mathvariant='normal'>T</mi>
 </msup>
</math>

Bottom up

1. $op:function is encountered. arg='op' is located.
2. <mi arg='op' intent='transpose'...> is processed and returns transpose.
3. Unwinding the stack, we have msup intent='transpose:function($arg)'>...
4. recursing on $arg gives x and so we have intent='transpose:function(x)' which gets spoken as "transpose of x"

Top down

1. $op:function is encountered. Because it has the function property, this is the same as laying this out as an mrow whose first element is the referenced $op, whose second element is an mo with &InvisibleFunctionApply, and whose third element is the referenced element $arg. That is

   <mrow>
   <mi arg='op'
     intent='transpose'
     mathvariant='normal'>T</mi>
   <mi arg='arg'>x</mi>
   </mrow>

2. We recurse and process the children. Since none of them have a reference, we just speak the resulting tree: "transpose of x".

Don't allow

There wasn't (I think) a good functional reason to reference a child that sets the intent. So the example could have been:

<msup intent='transpose:function($arg)'>
  <mi arg='arg'>x</mi>
  <mi  mathvariant='normal'>T</mi>
 </msup>

which to me is simpler and clearer.

More examples

I strongly encourage commenters to come up with an example of a higher order function that uses referenced elements in the head. Compare that to using apply for the head.

Also, I strongly encourage someone to come up with a use case where referencing a child in the function head has benefits over simply specifying the name on the parent element.

[dginev]

Example from Libre Texts, Decomposing Functions, eq (6.4.3), plain text (g∘h)′(x)

1. Maximal mrows, pushing intents as low as possible

   <mrow intent="$op3($x)">
     <mrow arg="op3" intent="$op2($composed)">
       <mrow arg="composed" intent="$op1:infix($g,$h)">
         <mo>(</mo>
         <mi arg="g">g</mi>
         <mo arg="op1" intent="function-composition">∘</mo>
         <mi arg="h">h</mi>
         <mo>)</mo>
       </mrow>
       <mo arg="op2" intent="first-derivative">′</mo>
     </mrow>
     <mo>(</mo>
     <mi arg="x">x</mi>
     <mo>)</mo>
   </mrow>

2. Minimal mrows, pulling intent as high as possible

   <mrow intent="first-derivative(function-composition:infix(g,h))(x)">
     <mo>(</mo>
     <mi>g</mi>
     <mo>∘</mo>
     <mi>h</mi>
     <mo>)</mo>
     <mo>′</mo>
     <mo>(</mo>
     <mi>x</mi>
     <mo>)</mo>
   </mrow>

[davidcarlisle]

@dginev thanks on the call we discussed HOF use cases but I didn't have a clear example to hand, this seems a good case.

On the call we discussed how if argref was banned as a function head you could make it the first argument of a core apply function as below, but I find your form 1 more natural

<mrow intent="apply($op3,$x)">
  <mrow arg="op3" intent="apply($op2,$composed)">
    <mrow arg="composed" intent="apply:infix($op1,$g,$h)"><!-- ?-->
      <mo>(</mo>
      <mi arg="g">g</mi>
      <mo arg="op1" intent="function-composition">∘</mo>
      <mi arg="h">h</mi>
      <mo>)</mo>
    </mrow>
    <mo arg="op2" intent="first-derivative">′</mo>
  </mrow>
  <mo>(</mo>
  <mi arg="x">x</mi>
  <mo>)</mo>
</mrow>

[NSoiffer]

I think all three examples over complicate the example and use intent where it's not needed. Of course, you can add it all over the place (including on the variables), but the primary goal is to disambiguate a notation. In this example, the only ambiguities are what ' and ∘ mean. The later is easily marked up on the operator itself leaving only one functional intent for derivative. So I think the simple markup is:

<mrow>
  <mrow intent='first-derivative($composed)'>
    <mrow arg='composed'>
      <mo>(</mo>
      <mi>g</mi>
      <mo intent='composed-with'>∘</mo>
      <mi>h</mi>
      <mo>)</mo>
    </mrow>
    <mo>′</mo>
  </mrow>
  <mo>&#x2061;</mo> <!-- apply function -->
  <mo>(</mo>
  <mi>x</mi>
  <mo>)</mo>
</mrow>

I don't see this as a case where we need a reference in the function head.

A couple of notes:

• The mrow nesting structure is copied from David's example -- I'd hope software would add more mrows such as around the parens.
• I believe the prime character should actually be in an msup, not an mrow. That matches what the author writes ($(g \circ h) ^\prime$) and is the obvious place for software to attach intent.
• When we get around to defining core names, we probably need to deal with the fact that there are different derivative notations and Newton's notation using "prime" doesn't specify the independent variable. In Leibniz's notation, we probably want a two argument function.

[dginev]

@NSoiffer If we had a magical invisible character available for every kind of intent-related phenomenon - and enough mrows - we could completely do away with "intent expressions" and use only simple concept annotations on the leaf token nodes:

<mrow>
  <mrow>
    <mrow>
      <mo>(</mo>
      <mi>g</mi>
      <mo intent='function-composition'>∘</mo>
      <mi>h</mi>
      <mo>)</mo>
    </mrow>
    <mo intent=":postfix">&#x2061;</mo> <!-- apply function -->
    <mo intent="first-derivative">′</mo>
  </mrow>
  <mo intent=":function">&#x2061;</mo> <!-- apply function -->
  <mo>(</mo>
  <mi>x</mi>
  <mo>)</mo>
</mrow>

Rather than focusing on "intent makes the example complicated", I think we should focus on the assumption that "invisible Unicode characters and mrows are an alternative for intent", which hasn't been discussed in sufficient detail.

Edit: added fixity properties Edit2: I keep that "encyclopedic names" are a good guide for choosing values, and function-composition fits that better than composed-with

[davidcarlisle]

If I understand @NSoiffer correctly that's more or less mathcat's internal world view: rewriting the intent expressions as extended mathml

[NSoiffer]

@dginev: The reason for this is issue is that complicated function heads are both hard to understand, and at least for me when implementing MathCAT, a significant pain point. It was also clear during the call today that many of us had different ideas of how they should be interpreted.

If they aren't needed, we can save implementers and ourselves time by not allowing them. But if they are needed, we need to resolve how they should be interpreted (the bottom-up/top-down overviews are two possibilities).

I'm still looking for an example where they really are needed. At the moment, I'm not even convinced that they are a good way to write intents, let alone a necessary way. But of course, "good" is subjective, so let's stick to "are they necessary?"

The invisible times in the example I gave is not needed. If it wasn't there, the outer mrow would need something like intent='apply($func, $x)', where func would be the first child. So again, no need for references in the head.

[dginev]

Sure, apply($func, $x) and $func($x) seem equivalent enough.

Is there a technical reason why one is easier to implement than the other?

[davidcarlisle]

The invisible times in the example I gave is not needed. If it wasn't there, the outer mrow would need something like intent='apply($func, $x)', where func would be the first child. So again, no need for references in the head

apply($f,$x) although it probably works, relies on apply being a known intent. In other issues we have said that terms should have reasonable default reading even if in core, "apply of f comma x" is probably understandable but not as nice as the default reading of $f($x)

similarly being in core rather than part of the gammar makes it trickier to define the behaviour of standard properties like postfix would that be apply:postfix($f,$x) or apply($f:postfix,$x) Neither seems totally natural to me and hard to specify from general rules about properties if apply is just "some core intent which may have system defined behaviour" rather than $f($x) which is a function call defined in the grammar so we can say much more about it in the spec.

[davidcarlisle]

The top down/bottom up discussion in the start of this issue needs to be decided (first) for all argref not just those possibly allowed in a function head.

The transpose example is too simple as the speech generated for the referenced element is transpose so does not depend on the outer property. As @brucemiller commented on the call such outer properties do (could?) have an effect on the way down, as speech for the referenced terms is generated, and on the way up, as the subterms are combined into the outer expression. :function only affects the way the terms are combine.

Suppose mtable had a :matrix property causing

"2 by 2 matrix, row 1 ...; row 2 .."

to be generated, and a :system-of-equations property causing

"equation 1 ...; equation 2..."

also assume the default reading is something else, say array rather than matrix

"2 by 2 array, row 1 ...; row 2 .."

if subterms are

<mtable arg="m1">
<mtable arg="m2" intent=":matrix">
<mtable arg="m3" intent=":system-of-equations">

and an outer element has intent="rank($m1:matrix)"

do you generate speech for $m1 "array..." and only consider :matrix "on the way up" while combining $m1 into the expression, where it is ignored as not affecting composition. So finally: "rank of array ...."

or do you generate speech for $m1 on the way down, in the context of the property :matrix so that rank($m1:matrix) is eqivalent to rank($m2) and makes "rank of matrix ..."

?

if you do rank($m3:matrix) is it the same as

<mtable arg="m3" intent=":system-of-equations:matrix">

and if so, what does it do?

I think all these questions are open and apply equally to argref in function heads or arguments, or at the top level since we allow intent="$m" so I don't think restricting argref from function calls simplifies much. There is a bias as most of our property examples come from @infix hints and only apply to function heads.

[dginev]

@davidcarlisle your last comment fits better in #449 . I am not sure that resolving that adds any clarity to the $ref function head question, which to me seems a separate design choice, even if there are no conflicting properties to discuss.

[davidcarlisle]

@dginev in a way yes, I suppose I was commenting that that should be decided first. While in principle function heads could be decided separately the description of the issue here is really all about the mechanics of expanding argument references, and that should be the same for argument references wherever they occur.

Incidently the derivative of function composition examples above can be seen in list 5 this is a slightly fake test as the implementation of mathcat being used isn't designed for this grammar, but it shows something and I'll update in place as the spec or implementation changes.

[NSoiffer]

Sure, apply($func, $x) and$func($x) seem equivalent enough.

Is there a technical reason why one is easier to implement than the other?

Let me start with an overview of what MathCAT does. It has three phases:

1. Cleanup the MathML (this is the most complicated and time consuming phase) -- result is MathML
2. Infer intent if not given -- result is an XML tree with heads that act as keys for speech (e.g., <binomial> ... </binomial>)
3. Generate speech (apply rules for known intents, else apply defaults)

As is maybe evident, when the head is complex, the matching algorithm that works for MathML no longer works for intent. My initial implementation of parsing intent didn't account for complex heads because there weren't examples of them (at least I didn't notice them when I wrote tests based on some examples). So one rewrite there. I didn't want to rewrite the pattern matcher previously used for MathML->Speech and now used for MathML->Intent. So the work around is to internally build an apply head. So extra work there plus extra work to realize that the head is special/different than the args. It was a source of bugs and between the second implementation and dealing with all the bugs accounted for 1/3 - 1/2 of the implementation time. The amount of extra code wasn't that great, but getting the tests right was a pain point.

Maybe a different implementation wouldn't have this problem. However, complex heads are harder (at least for me) to think about, so I strongly suspect they will be a source of errors for other implementations also. If there aren't compelling cases, why add the complexity?

[NSoiffer]

@davidcarlisle wrote:

similarly being in core rather than part of the gammar makes it trickier to define the behaviour of standard properties like postfix would that be apply:postfix($f,$x) or apply($f:postfix,$x) Neither seems totally natural to me and hard to specify from general rules about properties if apply is just "some core intent which may have system defined behaviour" rather than $f($x) which is a function call defined in the grammar so we can say much more about it in the spec.

Surely a core meaning doesn't make sense if :silent is added to it. I believe this generalizes to the simple rule: if you use a property previously known as a hint, then you have removed the core meaning. Hence, I don't think your concerns are a problem.

[NSoiffer]

@davidcarlisle wrote:

apply($f,$x) although it probably works, relies on apply being a known intent. In other issues we have said that terms should have reasonable default reading even if in core, "apply of f comma x" is probably understandable but not as nice as the default reading of $f($x)

That's a good point. Perhaps there is a better name that works with saying "of"?

[davidcarlisle]

@davidcarlisle wrote:

similarly being in core rather than part of the gammar makes it trickier to define the behaviour of standard properties like postfix would that be apply:postfix($f,$x) or apply($f:postfix,$x) Neither seems totally natural to me and hard to specify from general rules about properties if apply is just "some core intent which may have system defined behaviour" rather than $f($x) which is a function call defined in the grammar so we can say much more about it in the spec.

Surely a core meaning doesn't make sense if :silent is added to it. I believe this generalizes to the simple rule: if you use a property previously known as a hint, then you have removed the core meaning. Hence, I don't think your concerns are a problem.

I don't understand your answer at all, sorry:-)

If currently I have $f:postfix($x) and we stop allowing $ in the head and apply:postfix is no longer core (which I don't think should be true) then the only remaining suggestion is apply($f:postfix,$x) but would that work?

More abstractly I have problems with suggesting using apply($f,$x) as $f($x) they are only the same in the way 2 is the same as 1+1 or 3! is 6. If you are reading out function terms from a theorem prover, or lambda calculus etc, having to use an explicit apply from the meta-language for $f(x) but not for f(x) is pretty confusing,

In general I think the core list should be indexed by name, arity and fixity property/hint

so factorial is core and default postfix, but factorial:function($x) can still be core but choose a reading such as factorial of x, rather than x factorial

[brucemiller]

As is maybe evident, when the head is complex, the matching algorithm that works for MathML no longer works for intent.

Maybe I'm getting confused, but isn't it part of the point of the functional intent that you don't need any matching, since you're already being given the name, the arguments, and even in some examples the fixity (in many examples, you're given too many fixities!). In the cases we're discussing (referenced head, properties, etc), whether the name is in core or not won't be affecting any matching or search for arguments, only how the pieces are assembled into speech, right?

[dginev]

However, complex heads are harder (at least for me) to think about, so I strongly suspect they will be a source of errors for other implementations also. If there aren't compelling cases, why add the complexity?

Is this issue about references in the function head syntax, or is it about all "complex heads"? References are one special case, complex heads cover a lot more ground.

Some hypothetical examples of complex heads used without references:

• intent="first-derivative(f)(x)"
• intent="derivative(f,n)(x)"
• intent="inverse(sine)(x)"
• intent="laplacian(f)(x)" (aka laplace-operator(f)(x))
• intent="restriction(f,A)(x)"
  • we mentioned composition, another function transformation that needs a complex intent head is restriction
• intent="convex-conjugate(f)(x)"
  • written $f^*(x)$ see convex conjugate
• ...

One could argue for flattening all of them, as in derivative(f,n,x) or inverse-sine(x) but that will just move the awkwardness from the head into the concept names and/or argument list.

One could also argue for not having any way of representing these, but that will just move the issue to "surprises" in processing these cases realized via invisible apply (comment example).

I certainly agree these are harder cases than atomic functions and operators, and may be more difficult to implement and test.

[davidcarlisle]

I think if you have

intent=" ...$f ..." with a referenced <mi arg='f' intent='foo'>...

then $f should use the referenced intent so the effect should be

intent=" ...foo..." possibly needing scope intent=" ...(foo)..."

You should be able to do that whatever syntactic context $f occurs in, otherwise it's going to be hard to specify what an argref means at all.

As @dginev just commented, voicing higher order function applications is harder than functions with an identifier as the head but I can't see that using an argref makes it harder to specify.

I put one of Deyan's examples at

https://mathml-refresh.github.io/intent-lists/intent5.html#IDhofconvexconjugate

to see what happens currently.

The argref $fs($p) is handled the same way as the form with the $fs expanded, both with a slightly awkward but OK apply function of, prefix.

Actually the best reading is from the last form which has no argref and no intent funcalls at all, even though it does not recognise my speculative <msup intent=':decorated-function'> property to say msup wasn't a power (mathcat already avoids defaulting power when the superscript is *)

intent='convex-conjugate(f)(p)' gives apply function of, convex conjugate of, f comma p

"comma" is wrong here, actually as @NSoiffer indicated above it is reading apply-function(convex-conjugate(f),p) with no special rules to read that eg as infix: convex-conjugate of f applied to p

intent='$fs($p)' gives the same (so why ban it?:-)

<mo intent='convex-conjugate'>*</mo> gives f superscript convex conjugate, of p

[NSoiffer]

Bruce wrote:

Maybe I'm getting confused, but isn't it part of the point of the functional intent that you don't need any matching, since you're already being given the name, the arguments, and even in some examples the fixity (in many examples, you're given too many fixities!). In the cases we're discussing (referenced head, properties, etc), whether the name is in core or not won't be affecting any matching or search for arguments, only how the pieces are assembled into speech, right?

Being in core doesn't affect finding references, but it does affect how it is spoken. In my implementation, I have to be able to match the head and potentially the arguments and properties to know what to say for something that is in core. And I have to try to do the match to see if it is in core. At the moment, I think no one has proposed a core name that has a complex head, although maybe Deyan's comment above (intent="first-derivative(f)(x)", etc) is suggesting that. So a complex head may mean that AT wants/needs to generate some specialized speech.

[NSoiffer]

Deyan wrote:

Is this issue about references in the function head syntax, or is it about all "complex heads"? References are one special case, complex heads cover a lot more ground.

The issue is about references, but the pain point is about complex heads.

One could argue for flattening all of them, as in derivative(f,n,x) or inverse-sine(x) but that will just move the awkwardness from the head into the concept names and/or argument list.

I'm in favor of flattening them, but maybe that is because I don't know what the awkwardness is. Can you elaborate?

[davidcarlisle]

I still can not really see what the issue is here. If you need to recognise core names, and we have any kind of function syntax, you need to recognise names in subterms.

If plus is core, you can't just recognise intent="plus(a,b)" , you have to recognise both plus in intent="foo(plus(a,b), plus(c,d))"

The example you give is no different.

intent="first-derivative(f)(x)"

is a funcall with a non-core head, so process it with your standard funcall rules, at some point you will need to handle the subterm first-derivative(f) which is (for this discussion) a funcall with a core head which you can handle.

The processing would be the same with

intent="apply(first-derivative(f),x)"

you would still at some point have to handle first-derivative(f)

[brucemiller]

closeable?

[davidcarlisle]

yes, I think this was ressolved

[NSoiffer]

This diverged into a discussion of complex function heads. The original question needs to be returned to: how are references processed? So not resolved.

[davidcarlisle]

@NSoiffer I still can not really see a difference between references in a function head (this issue) and references at the top level or in arguments.

intent="$x"
intent="$x(a)"
intent="sin($x)"
intent="$x:prop"
intent="$x:prop(a)"
intent="sin($x:prop)"

all need similar handling as far as $x goes. What is true is that some properties only have an effect in some contexts

properties may affect the way speech is constructed for a term "on the way down" and also affect how that term is merged into the parent expression "on the way up"

so $x:infix does not have affect on the way $x is spoken but that text is placed in infix position in $x:infix(a,b) .

What is open here is

<mrow intent="$x(a,b)">
  <mo intent=":infix">+</mo>
</mrow>

I think it would be natural to consider that this infix is (eventually) used on a function head so end result is a plus b but I could live with specifying it has to be

<mrow intent="$x:infix(a,b)">
  <mo>+</mo>
</mrow>

Conversely if :system-of-equations on a mtable causes it to be read as equatons then either

<mrow  intent="$m">
<mtable  arg="m" intent=":system-of-equations">...

or

<mrow  intent="$m:system-of-equations">
<mtable  arg="m" >...

should work.

[dginev]

how are references processed?

Our reference+arg resolution algorithm is designed for a traversal starting from the <math> root, so consistency would imply top-down resolution. Namely, the arg attribute is always expected to be a descendant of the intent-carrying node that references it (via $arg). For navigation, the intent resolution process again starts from the root of a focused subtree (e.g. an <mrow>) and proceeds top-down.

We also expect that a valid referencing annotation always has two separate nodes associated with it: the node referencing via the $ syntax in its intent attribute, and the node being referenced via the arg attribute.

As long as neither node is discarded, forgotten or rewritten too early, the resolution should have a unique outcome. I would hope the main details to be clarified are the ones of property ordering ( in #449 ), which seems to be the only subtlety we've introduced.

Is there any other open question about "processing references" to discuss?

---

To comment on one curious example (slightly cleaned up):

<mrow intent="$op(a,b)">
  <mo arg="op" intent=":infix">+</mo>
</mrow>

Whether AT will decide to use the :infix property or not may indeed be an open question. But to me it seems clear that AT should have visibility of the property, after references are resolved. So while I am not sure if plus:infix(a,b) SHOULD be the required resolution, it seems to me that it MAY be a possible resolution.

If the inner intent of the node carrying arg="op" is inadvertently lost during resolution (i.e. :infix is dropped early), I think that should be seen as an implementation bug.

[NSoiffer]

It seems like everyone favors the top-down approach.

@dginev wrote:

For navigation, the intent resolution process again starts from the root of a focused subtree (e.g. an ) and proceeds top-down.

Since a parent can reference any descendant (unless blocked by another intent), not just the direct child, then navigation can land on some intermediate node where the intent is not specified. At least that is the case if someone is navigating the MathML tree, not the intent tree. Currently that's what MathCAT does although I can see a case for allowing navigation of the intent tree instead.

In the "curious example", landing on the mrow would give "a plus b" but navigating down the MathML just says "plus" and arrowing right/left say end/beginning of expression or something unrelated to the mrow assuming there is something else in the expression. It's probably not a good experience :-( It's also probably a reason that literal arguments should be discouraged unless absolutely needed.

[dginev]

Partial annotation leads to partial navigation, that seems unavoidable.

For full coordination between intent and the layout tree, one likely has to generate exhaustive and fine-grained intent annotations on each pmml node. Which is possible, while still optional.

It seems like we were mostly in agreement in the meeting as well, is this issue closable? Any pending decisions to be made?

[NSoiffer]

Closing issue because the primary topic (processing references in a function) seems to be resolved that it should be top-down.