Intent for "mtable" constructs

[dginev]

We have discussed some early variations of intent annotations for tabular constructs, thanks to Sam's prototype document.

I don't think we have a place to finish the discussion yet, similarly to the issue we made for n-ary operations in #253 .

There are at least three questions here:

• What do current AT tools struggle with in the tabular MathML examples we have, i.e. where do we need annotation beyond the baseline narration.
  • Preliminary: gather+systematize enough tabular examples.
• What would be an acceptable well-structured narration for each case?
  • Is there a default navigation pattern?
  • How does one prefer to navigate unusual walks in a tabular, from AT's perspective ?
• When we know what AT aims to produce, we can figure out appropriate values to use in intent, to guide the tabular walk.

---

Tonight I found a good non-standard example to add in: a MathML diagram. Such constructs are common in research documents that end up in e.g. arXiv.

The diagram can be found in MathML at ar5iv:2204.13136 and renders in Firefox as:

The latexml-generated MathML is the following (you need to expand the details tag):

Details

```html 𝒮= ⟨ a1 , a2 ⟩ ⟺ 𝒮⁢  is robust  ⇓ 𝒮⁢  has a unique Betti element  ⟺ 𝒮⁢  is generalized robust  ⇓ 𝒮⁢  is free  ⟺ in≺⁢ ( I𝒮 ) ⁢  is a C.I. for some  ≺ ⇓ I𝒮 ⁢ is a C.I. ```

This example is nice, because we can see a primary direction of implications going top-down on the left column. This connection should be possible to narrate clearly as a unified chained expression.

At the same time, the first 3 rows hold a standalone statement when read horizontally. The fourth does not.

The horizontal direction is similar to the common "aligned equation" case, with an infix relation aligned in the center column. The vertical direction is less usual / more interesting - it is realized with an empty row, holding an inked cell which contains the downward arrow, and two empty filler cells.

I hope the issue+example is helpful, feel free to edit/moderate/close as appropriate.

[dginev]

A second helpful example for "unusual" use of MathML tabulars I can offer is calculus syntax, written as what I would roughly describe as "a grammatical disjunction". An example is Figure 1 in ar5iv:1003.5513, rendered as:

As an initial thought, I think it could be helpful to mark up each pair of "expression"+"(description)" as associated (=to be read together), and collect all pairs into an n-ary operation - unsure with which name. As a first hand-waving guess, with $ for arg references:

<mtable intent="define($PQ, cases(label($e1, $output),label($e2,$input), label($e3,$nil), ...))"

The underlying MathML is:

```xml P, Q ::= u! ⁢ v→ . P (output) ∣ u⁢ ? ⁢ x → . P (input) ∣ 𝗇𝗂𝗅 (nil) ∣ 𝗂𝖿 ⁢ u = v ⁢ 𝗍𝗁𝖾𝗇 ⁢ P ⁢ 𝖾𝗅𝗌𝖾 ⁢ Q (match) ∣ 𝗋𝖾𝖼 ⁢ X . P (recursion) ∣ X (process variable) ∣ P ∥ Q (parallel) ∣ ( ν c : s ) P (stateful scoping) ∣ 𝖺𝗅𝗅𝗈𝖼 ⁢ ( x ) . P (allocate) ∣ 𝖿𝗋𝖾𝖾 ⁢ u . P (deallocate) ```

[NSoiffer]

What do current AT tools struggle with in the tabular MathML examples we have, i.e. where do we need annotation beyond the baseline narration.

In general, AT will announce it is a table and announce the rows and columns (usually the row # for the first entry and just the col # for the next columns). You should be able to navigate left/right and up/down columns, but I'm not sure if all AT can do that. Some might only have move left/right and when you get to the next col/row, it announces that.

Better AT will will recognize matrices and determinants and use that word in place of "table". Some might recognize "cases" (large "{") and use some appropriate wording for that.

People often use tables for alignment and that doesn't end up speaking well because the expression ends up being split with "col #" when reading/navigating.

MathPlayer and MathCAT allow you to mark a position and read/move to the position. That's useful for simultaneous equations, but in the ClearSpeak study we did (which didn't have those kind of examples), that feature was rarely used.

[dginev]

Example 3: Matrix with horizontal, vertical and diagonal ellipses

A more standard technique in matrix notation - elisions that imply an inductive pattern. Here we have an r-by-r size matrix, with empty cells for zero entries, and elisions for any adjacent cells where the respective indexes are offset by 1.

Viewable as a sub-expression in ar5iv:1502.07656, eq 3.12

I would imagine that when the construction is standard, authors would prefer intent values that are included in Wikipedia's List of named matrices. Although there is a case to be made that a single matrix value would suffice and the named cases can be provided via aria-describedby pointers, as auxiliary information.

Note that this example is not and "upper-triangular matrix" as the zeros are under the secondary/minor/anti- diagonal. I would assume if a name existed it would be in the spirit of "upper-anti-triangular matrix".

Rendered by Firefox as:

MathML source:

```xml μ 1⁢r ⋯ μ m-1 ,r μ m⁢r μ m+1 ,r ⋯ μ r-1,r μ r⁢r μ 1, r-1 ⋯ μ m-1, r-1 μ m, r-1 μ m+1, r-1 ⋯ μ r-1, r-1 ⋮ ⋮ ⋮ ⋮ ⋮ .⁢.⁢. μ 1, m+1 ⋯ μ m-1, m+1 μ m, m+1 μ m+1, m+1 μ 1⁢m ⋯ μ m-1,m μ m⁢m μ 1, m-1 ⋯ μ m-1, m-1 ⋮ .⁢.⁢. μ11 ```

aside 1: there are also many aliases for the two diagonals

aside 2: Equation 3.15 in this article has another non-standard construction, a GZ-pattern that uses an mtable with elisions, and is described at some length by the surrounding text.

[dginev]

Adding a comment both as a test and also to subtly hint to other group members following notifications here: today Github announced math support in markdown via TeX syntax rendered by MathJax.

I'd like to put more work into the authoring needs of Intent soon. The last example 3 here has the following "advanced typesetting" directive for the diagonal ellipsis (line-breaking mine, it's a single line in the source):

\def\qdots{\mathinner{
  \mkern1mu\raise1pt\vbox{\kern7pt\hbox{.}}
  \mkern2mu\raise4pt\hbox{.}
  \mkern2mu\raise7pt\hbox{.}\mkern1mu}}

Can Github handle that? $$ \mathinner{\mkern1mu\raise1pt\vbox{\kern7pt\hbox{.}}\mkern2mu\raise4pt\hbox{.}\mkern2mu\raise7pt\hbox{.}\mkern1mu} $$

Almost. If we dispense with the \vbox and its kern, we arrive at: $$ \mathinner{\mkern1mu\raise1pt\hbox{.}\mkern2mu\raise4pt\hbox{.}\mkern2mu\raise7pt\hbox{.}\mkern1mu} $$

Which renders. And - at least in my eyes - roughly looks like a diagonal ellipsis. Sadly no native MathML renderer option is available for now.

If we task latexml with creating some MathML here, it creates some nested <mpadded> scaffolding, with leaf content of three <mtext>.</mtext> single dot elements.

I'm tempted to ask a question of Neil (@NSoiffer), if I may: if encountered in the context of a matrix, would current AT read this ellipsis by extracting the text content? I.e. could we expect a baseline narration of "dot dot dot"? If so, interestingly, the baseline is already useful even without further annotation.

Still, the mathematically useful information is better conveyed by the name of U+22F0 (⋰) which is "up right diagonal ellipsis". Maybe the best remediation for this minimal example is to redo the \qdots macro in a way that maps into the Unicode character, and let the default narration for Unicode provide the upgrade. I'm usually not the one to advocate Unicode, but in this very specific case, it seems to be a good fit?

[NSoiffer]

First off, it's great to see github add this. Also great that you spotted it!

I just tried MathCAT, which is probably representative of several AT for this example, and it says "dot dot dot". As you say, that's not terrible, but better is "up right diagonal ellipsis" or "up right diagonal dot dot dot" which is what you get with U+22F0. So yes, for this case (and likely many others), using the Unicode character is likely the best thing to do.

[NSoiffer]

I meant to write earlier about some of the specific matrix shapes. I have a vague memory of saying this somewhere else (maybe on a call?). Currently, I don't think any AT recognizes special matrix forms such as a diagonal matrix, identity matrix, upper diagonal, reverse diagonal matrix as in your example, symmetric matrix, etc. Recognizing these and then not having to say all the elements would be a big time saver.

However, when learning about matrices, saying "3 by 3 upper diagonal matrix, ...". where "..." is a list of the non-zero elements might not be too helpful because the listener might not be familiar with the term. This is where AT having the ability to speak it differently depending upon user-preference/user-expertise would be helpful. However, I don't see these being in the "core"/known intents. At least not at the start.

[davidcarlisle]

sorry about previous comment, I shouldn't have read the thread on my phone:-) I didn't say anything not already said ⋰

[davidcarlisle]

@samdooley's examples were written assuming a more expressive intent syntax. I've tried to write them below using the version in the current draft.

The first version is more or less as in the original, just omiting the @ syntax. But I'm not sure if we are intending support argument-less intents such as intent="array" having dropped wildcards. So I also attempted a "fully arg-ed version. All the words are stolen from https://mathml-refresh.github.io/discussion-papers/intent#tables unchanged.

---

Examples -- Matrices and Tables

---

Matrices

The intent of an mtable may be used to construct a matrix.

$$ \left(\ \begin{matrix} 1 & 0 \ 0 & 1 \end{matrix}\ \right) $$

array(arrayrow(1,0),arrayrow(0,1))

<mtable intent="array">
  <mtr intent="arrayrow">
    <mtd><mn>1</mn></mtd>
    <mtd><mn>0</mn></mtd>
  </mtr>
  <mtr intent="arrayrow">
    <mtd><mn>0</mn></mtd>
    <mtd><mn>1</mn></mtd>
  </mtr>
</mtable>

or do we need

<mtable intent="array($r1,$r2)">
  <mtr arg="r1" intent="arrayrow($e1,$e2)">
    <mtd arg="e1"><mn>1</mn></mtd>
    <mtd arg="e2"><mn>0</mn></mtd>
  </mtr>
  <mtr arg="r2" intent="arrayrow($e1,$e2)">
    <mtd arg="e1"><mn>0</mn></mtd>
    <mtd arg="e2"><mn>1</mn></mtd>
  </mtr>
</mtable>

The intent of the table uses array, the intent of each row uses arrayrow and the intent of each element uses the default intent mrow.

---

Equation arrays

The intent of an mtable can be used to construct a list of equations.

$$ \begin{matrix} 2x &=& 1 \ y &>& x-3 \end{matrix} $$

list(append(times(2,x),eq,1),append(y,gt,minus(x,3)))

<mtable intent="list">
  <mtr intent="append">
    <mtd columnalign="right">
      <mn>2</mn>
      <mo>&#x2062;<mo>
      <mi>x</mi>
    </mtd>
    <mtd columnalign="center">
      <mo>=</mo>
    </mtd>
    <mtd columnalign="left">
      <mn>1</mn>
    </mtd>
  </mtr>
  <mtr intent="append">
    <mtd columnalign="right">
      <mi>y</mi>
    </mtd>
    <mtd columnalign="center">
      <mo>&gt;</mo>
    </mtd>
    <mtd columnalign="left">
      <mi>x</mi>
      <mo>-</mo>
      <mn>3</mn>
    </mtd>
  </mtr>
</mtable>

or

<mtable intent="list($r1,$r2)">
  <mtr arg="r1" intent="append($e1,$e2,$e3)">
    <mtd arg="e1" columnalign="right">
      <mn>2</mn>
      <mo>&#x2062;<mo>
      <mi>x</mi>
    </mtd>
    <mtd arg="e2" columnalign="center">
      <mo>=</mo>
    </mtd>
    <mtd arg="e3" columnalign="left">
      <mn>1</mn>
    </mtd>
  </mtr>
  <mtr arg="r2" intent="append($e1,$e2,$e3)">
    <mtd arg="e1" columnalign="right">
      <mi>y</mi>
    </mtd>
    <mtd arg="e2" columnalign="center">
      <mo>&gt;</mo>
    </mtd>
    <mtd arg="e3" columnalign="left">
      <mi>x</mi>
      <mo>-</mo>
      <mn>3</mn>
    </mtd>
  </mtr>
</mtable>

The intent of each element is constructed by treating each mtd element as an implicit mrow. So the first row contains times(2,x), eq, and 1, and the second row contains y, gt, and minus(x,3).

The intent of each row is constructed using append to combine the intent values from each element. So the intent of the first row is append(times(2,x),eq,1) and the intent of the second row is append(y,gt,minus(x,3)).

The intent of the table uses list to construct the list of equations.

---

Equation arrays

The intent of an mtable can be used to line break an equation.

$$ \begin{matrix} a &=& b + c - d \ & & + e - f \end{matrix} $$

list(append(a,eq,append(b,plus,c,minus,d),append(plus,e,minus,f)))

<pre>
<mtable intent="list">
  <mtr intent="append">
    <mtd columnalign="right">
      <mi>a</mi>
    </mtd>
    <mtd columnalign="center">
      <mo>=</mo>
    </mtd>
    <mtd columnalign="left">
      <mi>b</mi>
      <mo>+</mo>
      <mi>c</mi>
      <mo>-</mo>
      <mi>d</mi>
    </mtd>
  </mtr>
  <mtr intent="extend">
    <mtd columnalign="right"></mtd>
    <mtd columnalign="center"></mtd>
    <mtd columnalign="left">
      <mo>+</mo>
      <mi>e</mi>
      <mo>-</mo>
      <mi>f</mi>
    </mtd>
  </mtr>
</mtable>

or

<mtable intent="append($e1,$e2,$e3,$e4)">
  <mtr>
    <mtd arg="e1" columnalign="right">
      <mi>a</mi>
    </mtd>
    <mtd arg="e2" columnalign="center">
      <mo>=</mo>
    </mtd>
    <mtd arg="e3" columnalign="left">
      <mi>b</mi>
      <mo>+</mo>
      <mi>c</mi>
      <mo>-</mo>
      <mi>d</mi>
    </mtd>
  </mtr>
  <mtr>
    <mtd columnalign="right"></mtd>
    <mtd columnalign="center"></mtd>
    <mtd arg="e4" columnalign="left">
      <mo>+</mo>
      <mi>e</mi>
      <mo>-</mo>
      <mi>f</mi>
    </mtd>
  </mtr>
</mtable>

The intent of each element is constructed by treating each mtd element as an implicit mrow. So the first row contains a, eq, and append(b,plus,c,minus,d), and the second row contains append(plus,e,minus,f) since the other elements are empty.

The intent of each row is constructed using append to combine the intent values from each element. So the intent of the first row is append(a,eq,append(b,plus,c,minus,d)), and the intent of the second row is append(append(plus,e,minus,f)). Since the second row uses intent="extend", the intent of the second row is concatenated onto the intent of the first row to produce append(a,eq,append(b,plus,c,minus,d),append(plus,e,minus,f)).

The intent of the table uses list to construct the list of equations.

[dginev]

@davidcarlisle I added some TeX snippets to render before the XML examples above, which speeds me a bit with invoking the example in my mind. Might as well use the fresh new feature... until GH stops sanitizing away the raw MathML.

For the numeric matrix example, I also added the wrapping fences. I think for that case the intent ought to sit on a wrapping <mrow> which contains the <mo>(</mo><mtable>...</mtable><mo>)</mo> triple of siblings, as the fences are part of the construct, and AT may prefer not speaking fences when the contents are annotated.

I have some design thoughts on (avoiding) naming the pieces. But maybe they'll get more clear-cut when the three questions I itemized in my first message here get fleshed out answers on each kind of example we have.

[davidcarlisle]

@dginev yes I'd wondered if we should put TeX (source) if these are discussed in the spec, many people will be familiar with idioms for wrapping or aligning expressions in align or multline or gather ec. Not sure.

I didn't answer your questions about AT as really I'm not qualified to do so. Similary your more complicated examples I don't have suggested answers. Trying simpler cases first... Some are getting close to "commutative diagram layouts" that we've usually considered off topic for pure mathml. Alhough there is no clear dividing line. Of the three, probably the third is most clearly in scope as a pure mathml matrix of expressions

[dginev]

A very recent paper grabbed my attention today, by MIT's Visualization Group. It is titled: "Rich Screen Reader Experiences for Accessible Data Visualization"

You can find an accessible version (produced by latexml, even with a serving of natively-rendered MathML) here: http://vis.csail.mit.edu/pubs/rich-screen-reader-vis-experiences/

They performed a study with 13 "blind and visually impaired readers", focused on "data visualization and exploration" and offered 3 seemingly useful design dimensions that go beyond alttext:

structure, or how chart entities should be organized for a screen reader to traverse; navigation, or the structural, spatial, and targeted operations a user might perform to step through the structure; and, description, or the semantic content, composition, and verbosity of the screen reader’s narration.

Coincidentally, this fits rather directly with the way Neil has been presenting the motivation for "Intent", and - in the practical examples they go through - fits well with the motivating questions I am asking myself about how we should be annotating mtable-based constructions. Something to ponder...

[davidcarlisle]

the current spec has table row intent examples derived from @samdooley's CG document, lightly updated following comments above. So basically based on rather diffrent intent proposals than we currently have.

For example,

<mrow intent="matrix">
...
<mtr intent="arrayrow">

The current draft isn't as explicit as it will need to be about default readings, but my understanding is that by default the intent on the outer mrow will cause the entire expression to be read as "matrix" with the content of the actual matrix not used at all.

Presumably we could say matrix is in core and has some specified behaviour, but not sure what.. Earlier intent propsals had wildcards of various sorts but currently a non-core matrix construct would have to name and reference each of its rows.

I wonder if the hint feature can offer a way out with additional keywords so

intent=mymatrixthing @justreadhecontent

would allow you to tell the intent system to process the content without needing
mymatrixthing($r1,$r2,$r3,....)

This is far from being a fully worked out proposal, just a comment.

We need to put something in the spec even if it's only a note saying table markup will be addressed in a later draft.

[dginev]

We need to put something in the spec even if it's only a note saying table markup will be addressed in a later draft.

The "later draft" language sounds preferable to me to an artificial solution for the general case. But maybe a great solution will reveal itself.

We do have short cases in the discussion we've had here so far, which can be annotated purely conceptually, just for the sake of some examples:

• A small 2-by-2 numeric matrix could carry intent="matrix(row(0,1),row(1,0))"
• A small system-of-equations with 2 equations could carry
  • intent="system-of-equations($op1($lhs1,$rhs1)),$op2($lhs2,$rhs2))",
  • where the references point to the aligned via <mtd> pieces of each equation.
• A small single equation broken on two table rows, which gets its pieces glued back together either conceptually (if a grammar parse is possible), or via the _ or @silent technique for reading a list of pieces.
  • Grammatical: intent="equals(a,plus(b,minus(c,d),minus(e,f)))"
  • Pieces: intent="equals(a,_(b,plus,c,minus,d,plus,e,minus,f))"
  • I wrote them literal for clarity, but each of the literals can also be a $ref to the Presentation MathML token element that holds the content (mi or mo in this case).

Edit: I would see matrix and system-of-equations as part of Intent Core, and maybe another handful of standard K14 concepts that end up tabulated in presentation.

[davidcarlisle]

A small 2-by-2 numeric matrix could carry intent="matrix(row(0,1),row(1,0))"

Yes that's the question really.

Are we going to stick to the status quo which is an intent function needs to enumerate its arguments so the current spec example intent="matrix" needs to be intent="matrix($row1,$row2)" with the <mtr having arg="row1" ...

Or are we going to bring back some form of wildcard, by special-casing matrix, or adding back some wildcard syntax, or by finding an inventive use of @hint ?

I 'm just asking the question, I don't have a good answer, but as a general principle I'd rather the spec was self-consistent, even if we plan on changing it.

So I think (?) at least for now, the example (simple 2x2 (1,0,0,1) matrix) at https://w3c.github.io/mathml/#mixing_intent_examples_mtr

<mrow intent="matrix">
  <mo>(</mo>
  <mtable>
    <mtr intent="arrayrow">
      <mtd><mn>1</mn></mtd>
      <mtd><mn>0</mn></mtd>
    </mtr>
    <mtr intent="arrayrow">
      <mtd><mn>0</mn></mtd>
      <mtd><mn>1</mn></mtd>
    </mtr>
  </mtable>
  <mo>)</mo>
</mrow>

should be

<mrow intent="matrix($r1,$r2)">
  <mo>(</mo>
  <mtable>
    <mtr arg="r1" intent="arrayrow($a,$b)">
      <mtd arg="a"><mn>1</mn></mtd>
      <mtd arg="b"><mn>0</mn></mtd>
    </mtr>
    <mtr  arg="r2" intent="arrayrow($c,$d)">
      <mtd arg="c"><mn>0</mn></mtd>
      <mtd arg="d"><mn>1</mn></mtd>
    </mtr>
  </mtable>
  <mo>)</mo>
</mrow>

I would see matrix and system-of-equations as part of Intent Core,

Yes I think these should be in core.

[davidcarlisle]

We eventually dropped wildcards (and path selectors) from earlier intent proposals due to the difficulty of deciding what would be the default elements selected and are elements children or grandchildren if surrounded by "redundant" mrows. But (since mathml2) mtable is very constrained, it is just a sequence of mtr each of which is a sequence of mtd.

As such I think we ought to be able to have

<mrow intent="matrix($m)">
  <mo>(</mo>
  <mtable arg="m">
    <mtr>
      <mtd><mn>1</mn></mtd>
      <mtd><mn>0</mn></mtd>
    </mtr>
    <mtr>
      <mtd><mn>0</mn></mtd>
      <mtd><mn>1</mn></mtd>
    </mtr>
  </mtable>
  <mo>)</mo>
</mrow>

where the intent concept function matrix takes the argument $m, which meams basically "read the referenced mtable as a matrix) and you don't have to enumerate all the entries.

This touches on something @dginev mentioned during yesterday's call, that we need to specify some default behaviour for elements without intent, here mtable/mtr, but more generally too.

[davidcarlisle]

@dginev

wrote them literal for clarity, but each of the literals can also be a $ref to the Presentation MathML token element that holds the content (mi or mo in this case).

But can it? In a mtable that has two equations each wrapped over two rows, there is no container other than the mtable so unless you have a single intent for the entire construct you can't have an intent that just glues two rows together. To do that we would need to add an <mtgroup or <mtbody or some such to group table rows (like html <tbody>) or we need to allow $ref to reference a sibling (or descendents of a sibling)

the current spec has <mtr intent="append"> and <mtr intent="extend"> but with intent as now defined this would need to be <mtr intent="append(?)"> and <mtr intent="extend{?)">

I'd like to put

intent="equals($a,plus($b,minus($c,$d),minus($e,$f)))"

on the first row even if arg=e and arg=f are entries in the next row, but if that is allowed, what is the scope of these names?

we could say, that to resolve $ref you first search descendants then you recursively search descendants of ancestors until you find a matching arg=ref, but we don't say that currently.

[dginev]

there is no container other than the mtable so unless you have a single intent for the entire construct you can't have an intent that just glues two rows together.

I think this is a fair summary, and an acceptable restriction in practice. Rows that are conceptually self-contained can get annotated on their <mtr> and multi-row constructs will have to get annotated on some common ancestor, the nearest being <mtable>, if we insist on using references. If we expand the references, there is nothing preventing us from using the content of row 3 as a literal annotation on row 2.

Edit: And one extra thought about the rigidity of MathML tables. You say "mtable is very constrained", but have you considered the ability to use the columnspan and rowspan attributes in MathML table cells? They allow breaking out of the grid metaphor in a rather open-ended extent. And while most tables will be simple grids, use of the spanning attributes will (at least in some particular constructions) require an mtable-level annotation to structure a correct "intent" value.

[davidcarlisle]

think this is a fair summary,

I was hoping you'd say I'd missed something

and an acceptable restriction in practice.

I suspect it makes it pretty hard to use in practice. If you have a marked up alignment then decide to wrap one row, you have to change the intent markup for the whole alignment.

but have you considered the ability to use the columnspan and rowspan

Yes I considered mentioning them but decided I'd raised too many questions already for an initial stage. But since you ask, at least the intent=matrix($m) suggestion, has the advantage of defering to "read the matrix entries" so as long as the system can read a spanning entry, that does not impact on the intent markup.

[dginev]

@davidcarlisle I can see the simple matrix($m) value as usable from the wrapping mrow, given that the matrix fences (be they parentheses or brackets - I've seen both), are not in the mtable element.

So the following may be expected to already produce reasonable outcomes with AT, if matrix is in Intent Core:

<mrow intent="matrix($m)">
  <mo>(</mo>
  <mtable arg="m"> ... </mtable>
  <mo>)</mo>
</mrow>

Edit: Actually yes, this is exactly what you said in your comment. So I'm on board.

[NSoiffer]

Hopefully I didn't miss anything (written while jammed into an airline seat and later on a bumpy bus): this discussion assumes for the sake of discussion that matrix, etc, is not in core. However, all MathML elements have a default reading (because one can't guarantee intent is used), so mtable has a default reading.

ClearSpeak has options for controlling speech in tables. The options include "lines", "equations",  and "cases". E.g, "3 lines, line 1, 2x+y, line 2, 3x -y". Getting that kind of speech, or more complicated speech where column numbers are spoken also gets ugly but can certainly be done with the current version of intent by using _. Because it is so much work, it is probably best done by software generated output.

First off, I really don't like the idea of allowing argref's to look outside of their children. That makes the search unconstrained, makes it harder to find errors, etc. I agree with Deyan that the unfortunate but unreasonable requirement that stitching together two rows requires writing a (likely) complex intent on mtable.

The matrix markup in

<mrow intent="matrix($m)"> 
   <mo>(</mo>
  <mtable arg="m">
     ...
  </mtable>
  <mo>)</mo>
</mrow>

is probably ok because a default reading of mtable almost certainly involves saying the row and column numbers. So "matrix of row 1, xxx, ...". It is not ideal and the "of" can be overridden or if sensible, AT could know not to say "of" when the argument is a matrix. I'm sure Deyan can come up with examples where saying "of" is appropriate with mtable, so maybe AT can't do better.

If we have equations that are aligned (and therefore spread out among columns), then without some sort of * notation to indicate "take all the children", I don't see any solution other than listing all the columns with a silent hint or at least partially reconstruction the expression layout (equal(..., ...)). Although I don't object to a *-like syntax addition, I'd rather not add it at this time. I don't think it would be much work for a TeX-macro to generate all the argrefs and arg=xxx. It's advantage would mainly be for humans.

Throwing two ideas out in case they trigger some better idea:

1. What if we say that a function with no arguments acts like it takes all the children? We were hard-pressed to come up with useful examples of no-arg functions, so repurposing that.
2. What if an argref that points to itself indicates that all the children should be joined? (i.e., `... intent='$row' arg='row')

Alternatively, but still not pleasing, we can add a "special" known functions ("append"?) which take no arguments but are defined to gather all the children of the element and then say nothing (or something).

I don't see a "clean" solution to get row/line/equation/case to be spoken without a liberal use of _ or saying they are part of core (and hence AT can add the words the appropriate wording). And I don't see a simple way of gluing columns together without listing them all without adding syntax.

[dginev]

@tekemustafa Merhaba komşu,

I see that you translated a couple of (Neil's) comments from previous discussions to Turkish and posted them as new replies. I am not sure why? In any case, the official discussion language in this github repository is English, so I will delete them for now. Feel free to give us feedback on any of the issues in English, if possible.

[dginev]

Something to track (and potentially reach out) is the CSS effort to allow annotating layout navigation order for accessibility (specifically in the new flexbox and grid cases). The working name is reading-order.

There is a recent call for feedback with more details here: https://developer.chrome.com/blog/reading-order/?ref=csslayout.news

[brucemiller]

To summarize (from my POV): this issue may be closable, modulo a few clarifications in the spec.

A single intent expression at (or near) the mtable is capable of capturing any of the discussed structures in the general case, ideally making heavy use of references to table cells, or their contents. This form is quite heavy and cumbersome, however.

At the other extreme, an appropriate set of properties, again applied at or near the mtable, can capture some of the more common situations, such as :array, :equations, :cases, perhaps even :diagram. We should probably focus on a small set initially, with the expectation of augmenting the set after user experience. The set of properties would of course be open.

The in-between cases which aren't easily captured by a property, but which are too cumbersome to write in a single full intent is tricky. I'm hesitant to recommend a bunch of lispy special case keywords. I might find a single special case reasonable: For example mrow($a,$b,$c) might mean to combine the contents of the referenced things into a single mrow, giving it the intent attribute being the first(?) intent found on the referenced things. Would this cover enough additional situations to be worth the confusion?

[davidcarlisle]

use of properties allows us to close this