Lookup direction flags

[behdad]

Currently lookups are applied in logical forward direction. Use two (from newly-to-be-made available https://github.com/be-fonts/boring-expansion-spec/issues/26) LookupFlag bits to add three mode bits for the following directionalities:

• logical, backwards,
• visual, ltr,
• visual, rtl,

This is similar to how morx table works in fact: Bit 30	Bit 28	Interpretation for Horizontal Text
0	0	The subtable is processed in layout order (the same order as the glyphs, which is always left-to-right).
1	0	The subtable is processed in reverse layout order (the order opposite that of the glyphs, which is always right-to-left).
0	1	The subtable is processed in logical order (the same order as the characters, which may be left-to-right or right-to-left).
1	1	The subtable is processed in reverse logical order (the order opposite that of the characters, which may be right-to-left or left-to-right).

Except that the default in OpenType is logical... So, the bits will be: "visual", and "reverse".

If the two bits combined result in a direction opposite of the current buffer direction, the buffer contents are reversed before lookup application, lookup applied, then buffer contents reversed back to their original direction. The application, obviously, can be optimized and does not have to follow the strict reversal procedure.

This general lookup-direction model solves many problems. For example, emoji and Arabic numerals currently need special code in the shaper because the have direction/script properties that make it hard/impossible to write robust lookup rules for them. These flags allow that to be addressed.

Another thing this fixes is extending the features of Reverse substitutions to be as powerful as all substitution types. Currently ReverseChainSingleSubstFormat1 can only do one susbtitution, so a complex font like Gulzar has over 3800 lookup subtables like that. That's insane! Using these lookup flags, a ChainContenxtFormat2 using classes can be used to optimize that for example.

[nedley]

I would say this is still fairly important, especially for characters like punctuation that may occur in either visual order. Not sure this requires more bits, though: my strawman suggestion would be to treat the existing RIGHT_TO_LEFT LookupFlag (bit 0) as “reverse” for all types and use one of the reserved bits for visual.

I’ve lost track of where things stand with versioning at this point, can we introduce this with the new lookup types?

[behdad]

I would say this is still fairly important, especially for characters like punctuation that may occur in either visual order. Not sure this requires more bits, though: my strawman suggestion would be to treat the existing RIGHT_TO_LEFT LookupFlag (bit 0) as “reverse” for all types and use one of the reserved bits for visual.

That's certainly appealing. But it would have introduce an inconsistency. The way I like to spec / implement this feature is that if the buffer direction doesn't match the lookup direction, then reverse the buffer and apply lookup. That wouldn't work with CursivePos if we follow your proposal, since the Entry/Exit anchors should also be reversed to maintain the current behavior.

I think two new bits would work more consistently. Also I have a vague memory of seeing RightToLeft bit set on non-Cursive lookups in Arabic fonts.

I’ve lost track of where things stand with versioning at this point, can we introduce this with the new lookup types?

I think we can add it to all lookups without any version bump. That has been done before with MarkFilteringSets I believe.

[nedley]

My thinking here was to broaden the use of the flag beyond CursivePos but you raise a good point in allowing direction flags to affect it too.

I think we can add it to all lookups without any version bump. That has been done before with MarkFilteringSets I believe.

MarkFilteringSets bumped minor GDEF version to 2, IIRC.

[behdad]

MarkFilteringSets bumped minor GDEF version to 2, IIRC.

Yes, because a new field was added to GDEF header. My point was that GSUB/GPOS versions weren't bumped just because of the new flag, and I don't recommend doing them for this either.

[nedley]

My concern is that without a version break old clients might get dramatically different layout, making it difficult to deploy fonts using the new flags.

[behdad]

A major version bump is heavyhanded here. A minor version bump would not help old clients.

[behdad]

Unless we duplicate all lookup types with new lookup-type numbers...

[nedley]

So where does that leave us? Should the new flags only be valid for new lookup types?

[behdad]

I expect that we add the flag, and in a few years it can be used widely. That's like pretty much any other "minor" update.

This feature mainly applies to emoji sequences, Arabic subtending marks, plus allowing "Reverse" version of all lookup types. NotoColorEmoji for example ended ligatures for emoji sequences in both directions to work around this problem before...

That is to say: adding the flags now makes old systems no worse. And I don't see how this is any different than eg. when USE was introduced.