Create explainer for Enhanced TextMetrics

[AndresRPerez12]

Explainer for the Enhanced TextMetrics proposal that adds getSelectionRects() and getActualBoundingBox() to the API. Once the feature is submitted under a flag in Chrome, it will be updated with the flag name and instructions to use it.

[Kaiido]

IIUC measureText() still works only on a single line of text at a time, right? Then in what circumstances is it expected that getSelectionRects() could return multiple rects?
Which also brings the question of the integration with https://github.com/WICG/canvas-formatted-text. I'm not totally on page on that proposal, but wouldn't that API somehow supersede measureText()? And as such, wouldn't it be better for these new methods to be based on that API?

[fserb]

Which also brings the question of the integration with https://github.com/WICG/canvas-formatted-text. I'm not totally on page on that proposal, but wouldn't that API somehow supersede measureText()? And as such, wouldn't it be better for these new methods to be based on that API?

We are updating WICG Canvas Formatted Text soon (this week or next) and it will include this proposal and more. The high level plan includes keeping measureText() with a single text run (i.e., a single line with one font style) as a way to get metrics in a faster way, including some metrics that are currently only available in the DOM (like selection range).

[AndresRPerez12]

IIUC measureText() still works only on a single line of text at a time, right? Then in what circumstances is it expected that getSelectionRects() could return multiple rects?

Yes, it is correct that measureText() is meant only for single lines of text. Multiple rects can still happen in this case for bidirectional text, since adjacent characters in the input string might not be adjacent visually after the reorder.

Also, the objective is to match what the DOM does when the selection is queried via getClientRects(), and for bidirectional text a rectangle is returned for each logical run (the result of splitting the input text when the direction changes), even if the complete logical run is selected. So even if visually it looks like a single rect, if the selection spans several logical runs, getClientRects() will return several rects.

[Kaiido]

Thanks for the precisions. These bring a couple more questions that I'll write down here, merely to not forget later on and may not require answers at this stage.

Multiple rects can still happen in this case for bidirectional text

This means measureText() would run the bidi algorithm. It's really not my domain of expertise, but IIRC that used to be a big interop issue, wasn't it? Looking at the specs today it seems they just don't talk about bidi at all and only take into consideration the context's direction attribute... Then I suppose this brings the question of what would happen for something like

const text = '\u202dالخط العربي bar.';
const tm = ctx.measureText(text);
const range1 = tm.getSelectionRects(0, 5);
const range2 = tm.getSelectionRects(1, 5);

Since it's said the range operate on characters, I assume the invisible left override character counts in the range selection. But should it still have an effect on range2 or is a new bidi pass done every time?

And I guess the TextMetrics objects will become a lot heavier too, they should at least keep a reference to the input, and maybe even to the font styles of the context?

[AndresRPerez12]

Thanks for the questions! I'll answer with regards of our current approach in Chrome:

This means measureText() would run the bidi algorithm.

We are already running it for measureText() since these logical runs are needed to correctly measure and calculate the metrics of the input text for the bidi case. For the new functions, it would be necessary to keep some information from this process that wasn't being saved before in order to answer the new queries on the TextMetrics object. What is saved exactly depends on what we want to pursue in terms of the memory/speed tradeoff, and how much we want to pre-calculate from the moment measureText is called.

Since it's said the range operate on characters, I assume the invisible left override character counts in the range selection. But should it still have an effect on range2 or is a new bidi pass done every time?

Interesting question! Our approach for now is that everything is measured/calculated with the whole input text as reference. So only an initial bidi pass is done when measureText is called, and everything else is calculated from the logical runs obtained from there. So in this case, the directional override applies in both cases, and does count as a character for the range selection.

And I guess the TextMetrics objects will become a lot heavier too, they should at least keep a reference to the input, and maybe even to the font styles of the context?

We will have to save more data, but the amount is dependent on the implementation approach. It's possible to measure everything from the moment measureText is called and after that just join rectangles, but this would make measureText much slower. By saving the font and the logical runs it's possible to do most of the calculations just when the new API is called; but yes, this will mean a heavier TextMetrics object.

By the way, we are also in the process of removing some data from the TextMetrics object that we were keeping from previous attempts at expanding its functionality. That would offset at least part of that difference in our case.