Caret rendering and clicking

[stefnotch]

For rendering a caret:

• [x] Zipper + offset => physical location

To get such a physical location, we could walk down the tree until we reach the desired location.

For clicking in the formula

• [x] Click on a physical location => Zipper + offset

For the user

• [x] Highlight the element/container where the caret is

To get such a zipper, we could walk down the tree, keeping track of where we are.

Walk down the tree: The tough part here is that the MathLayout doesn't really correspond all that well to MathML. One possibility would be to always re-render for those operations 🤔

[stefnotch]

Note that language servers have a super similar problem

from https://d-nb.info/1180388712/34 page 24

[stefnotch]

Map<HTMLElement, PartialRange>

/**
* Pretty please make sure they don't overlap and stuff.
*/
PartialRange {
  zipper: PartialZipper
  from: Offset;
  to: Offset;
}

/**
* You gotta look at the parents to figure this one out
*/
PartialZipper {
  // TODO: maybe number[] as in 'index in parent'
}

I could also store direct references to the zippers, but the zippers are rather expensive and designed to be created lazily. So this design that lets one reconstruct a zipper is superior.

[stefnotch]

I think I can build a pretty good tree traverser that also keeps track of the whole range/partial zippper stuff.

It'd be a combo of the tokenstream and a zipper

[stefnotch]

Is storing all caret positions (x, y coordinates) a good idea?

• no, because reading such info from the browser is a bit expensive https://toruskit.com/blog/how-to-get-element-bounds-without-reflow/#boundingclientrect-to-the-rescue : 3ms to 5ms , including the offsetLeft and offsetTop options

[stefnotch]

One case that mostly serves to demonstrate the tricky-ness of mapping between the MathLayout and the MathML is

{ type: "row",
values: [ {type:"symbol", value: "3"},{type:"symbol", value: "."},{type:"symbol", value: "1"},{type:"symbol", value: "4"}, ]
}

<mrow>
  <!-- Digits get smooshed together into one number -->
  <mn>3.14</mn> 
</mrow>

And even worse ["(", "a", "+", "b", ")"] would become something like <mrow><mo>(</mo> <mrow><mi>a</mi> ...+b...</mrow> <mo>(</mo></mrow>

So how about...introducing a Treenslator tree, that lets you translate between either representation. Basically, it's a tree that mirrors both the MathLayout and the DOM. Well, it mostly mirrors the MathLayout and then stores where that is in the DOM.

• mirrors the MathLayout and then stores where that is in the DOM: So "Zipper + offset => physical location" becomes easy, just walk down a tree in log(n) time. Notice how only rows and texts need to be able to give you a physical location. The other elements, like the fractions, are purely there to make the tree traversal easier.
• "Click on a physical location => Zipper + offset" involves walking up the DOM and noting down every single DOM parent in log(n) time. Then we walk down the tree, each time attempting to find the correct node which has one of the DOM elements. This is trickier, we need more guarantees here.

The alternative is treating every character as its own node. And since we can't always attach info to them, we'll do some smort guesswork.

[stefnotch]

One useful guarantee might be:

• Every MathLayout ROW node corresponds to a specific DOM subtree, and nothing more. So with just a given MathLayout node, one can always reconstruct said subtree.
  • A MathLayout symbol node can't have that guarantee, because 123 is 3 symbols, but one <mi>123</mi>

[stefnotch]

When you have a row-zipper + an offset and you ask "where is the caret position in the viewport, and how tall should it be", then

• we can get the corresponding DOM row
• the x positions of the child elements are used for the caret x positions
• the y position is obtained by getting the row's baseline
• the height of the caret is obtained by asking the row how tall text in it is, see also https://github.com/w3c/mathml-core/issues/38

Some other, discarded options regarding that were

// - every child has a "bounding box" (x, y, width, height) -> we can get positions, even the final position
    // vs
    // - every child has a "before-position" (x, y) and a "height" -> we can get positions, but not the final position
    //   for the final position, we add a "final element" or something. This also handles the case where the row is empty.
    // vs
    // - every child has a "before-position" and "after-position", cause the height is the caret height!
    //   And the position.y is just where the caret should be, not how deep the element is

[stefnotch]

It can do stuff now. Zipper + offset => physical location almost works.

[stefnotch]

I guess pointing the performance profiler at it would be interesting. Especially given how ugly the baseline getting is.

[stefnotch]

"Click on a physical location => Zipper + offset" involves walking up the DOM and noting down every single DOM parent in log(n) time. Then we walk down the tree, each time attempting to find the correct node which has one of the DOM elements. This is trickier, we need more guarantees here.

An alternative would be: Every DomTranslator has an element that definitely physically contains all of its children. And then we repeatedly calculate the shortest distance to all possibly relevant elements. See also https://github.com/arnog/mathlive/blob/9481cfbc6229deff07838f84c4350640f4b1676c/src/editor-mathfield/pointer-input.ts#L337

[stefnotch]

For completeness, some alternatives to the DomTranslator tree would be

• There's the option of throwing away the rendered->zipper info and re-obtaining it when needed by rerendering the formula. Having a 60 FPS renderer isn't an unreasonable requirement. Ideally, it could do some caching though.

• I could also split the emitting into "emit" and "iterate in lockstep"

[stefnotch]

Works nicely now