cursor position, leading/trailing and affinity

[cmyr]

I ran into an issue in piet-coregraphics where hit_test_position wasn't respecting trailing newlines, and this has led me down a bit of a rabbit hole. In particular, I'm curious that there doesn't seem to be any concept of leading/trailing or upstream/downstream here? Is this an oversight, or am I missing something?

As an illustration, imagine the text "AAA BBB" with a soft break, so lines are "AAA " and "BBB". If I hit-test position 4, both "AAA |" and "|BBB" are valid. Is there some trick I'm missing? This doesn't seem to be as much of an issue on piet-d2d, although I haven't been able to play around there quite as much.

I'm pretty sure I could implement this in piet as some kind of cursor_pos_for_point method, because then I could ensure I take the line of the point into account when choosing my position instead of resolving to a string index and resolving from that back to a point, which is lossy?

[raphlinus]

Suuuuper good question.

Long story short, the best way to do this is probably to plumb affinity into the hit test method. I think "position + affinity" is the cleanest high-level representation of the concept. Also note that affinity is closely related to the primary/secondary distinction in BiDi.

I found it very interesting to see how DWrite deals with this question. There, it's not really a question of "position" and "point," but rather the model represents text as grapheme clusters with finite width. Each grapheme cluster then has a leading and a trailing position, and the ordering of that reflects BiDi direction. In your example above, "AAA |" is represented as the trailing position of the space, and "|BBB" is represented as the leading position of the first "B". One of the significant advantages to this model is that in a BiDi context, each grapheme cluster has a definite direction, while a "point", being in general a boundary between two grapheme clusters, can be ambiguous in this way at direction boundaries, hence the primary/secondary distinction.

The "grapheme cluster has finite width" approach is also an extremely clean mental model for the selection region issue: the selection region is simply the union of the rects defined by the grapheme clusters within the selection.

One approach is to adopt the D2D approach, and shoehorn other APIs into it. But, as I mentioned, I think point + affinity + maybe pri/sec better captures the higher level use case. If we design an interface for this, we should probably at least be thinking about BiDi implications. I actually think I could work this out in about a half hour of thinking/experimenting/writing.

[raphlinus]

Ok, we discussed this thoroughly on Zulip, and here's my synthesis:

First, while affinity is a very nice item of polish, it doesn't feel necessary to me in this cycle. I suggested taking it in the same cycle as BiDi, because it touches many of the same points of platform integration.

Second, I think I have the answer to how affinity intersects with the primary/secondary distinction in BiDi. Let me phrase that in terms of the DWrite concept of leading/trailing.

If a position in logic text (ie an offset in code units) is not a line break, then the mapping between logical position and physical position is not sensitive to affinity. The mapping is as follows: if the logically-before run is the same direction as the paragraph, then the physical position is the trailing position of the logically-before run. If the logically-after run is the paragraph direction, then it is the leading position of the logically-after run. (When the directions are the same, ie it is not a BiDi boundary, these physical positions should coincide). This assumes "primary." For "secondary," replace paragraph direction with its opposite, ie there's an "xor"-like logic.

At a line break, the mapping is determined by affinity. For upstream affinity, the physical position is either the leading or trailing position of the logically-before run. It is the trailing position if that run has paragraph direction, otherwise leading. Similarly for downstream, it is the leading position of the logically-after run if that run matches paragraph direction, otherwise trailing.

Note that the "reset" logic in the BiDi algorithm always assigns paragraph direction to trailing whitespace before a line break. The logic as stated above should be based on the computed direction after applying reset.

The hit_test_point method can always assume "primary", and should report affinity as well as string offset. To support affinity properly, the hit_test_text_position should take an additional affinity argument. To support BiDi, it should also take a primary/secondary toggle. The main motivation to query secondary position is rendering an Apple-style split cursor, and it's not obvious we need to do that; my experimentation with more recent text stacks suggests it may no longer be a requirement. We need feedback from BiDi experts on this.

The current mapping of the isTrailingHit result from the DWrite HitTestPoint method is not BiDi aware. I believe to get this right will require iteration over runs. But all this is for a future cycle, I'm just writing that here for reference.

[cmyr]

This has come up again, in response to a bug reported in zulip during the review of https://github.com/linebender/druid/pull/1636.

Basically: the ability to correctly select the entire length of a line that has an "emergency break" (that is, where there is no whitespace available to break the line at, and so we break it between two normally non-breaking characters) doesn't work without affinity, because we can't find the correct offset for the last character in the line; it always resolves 'downstream', which means it resolves to the start of the next line.

For illlustration, here is a broken selection:

https://user-images.githubusercontent.com/3330916/110377012-e0e03c00-8021-11eb-9c00-e3421f6f5b63.mov

each time the selection covers an entire 'emergency broken' line, that line ends up with no visible selection region.

In any case: there's a hacky way to work around this for now, specifically for selections, which is to manually check when a selection region ends at the end of a line and that line doesn't have trailing whitespace; when this is true it means we can just clamp the selection to the width of the layout.