Layer rendering order across glyphs (“`XOLR`” table)

[ctrlcctrlv]

(Loose continuation from https://github.com/BlackFoundryCom/black-renderer/pull/65)

Beginning with version 2, I will begin writing into the color fonts of FRB American Cursive a table I am calling XOLR.

XOLR — Color font representation requirements

XOLR_FORMAT = """ 
    > # big endian
    version:     L
    flags:       L
"""

  <XOLR>
    <hexdata>
      00000000 00000001  
    </hexdata>
  </XOLR>

I understand no one supports this table. At first only my own tools will use it.

I intend to start discussion of the necessity of such table or spec amendment to say, how to composite layers across glyphs.

XOLR will define these flags for bits 1–2:

№ 87654321
0b00000000

0b0 — traditional method, draw glyphs layer-by-layer, glyph-by-glyph (equivalent to NULL XOLR table) 0b1 — cursive method, draw glyphs layer-by-layer, run-by-run (rasterizer must be able to delay drawing until end of run) 0b10 — draw glyphs layer-by-layer, line-by-line (rasterizer must be able to delay drawing until end of run) 0b11 — draw glyphs layer-by-layer, page-by-page (or document-by-document in non-paginated contexts, the strictest joining behavior, rasterizer must have full control of document)

Bit 3 will be used to mean "you may need to support doing unions as well". Think about an Arabic color font at ½ opacity.

At 1.0 opacity no union would be needed, so the rasterizer wouldn't need a booleanOperations.

FRBAC has flags value 0b010. (lines should never collide in normal use, no need for a union as all layers are 100%).

[rsheeter]

+1 to the horrors of transparent overlapping shapes! See https://github.com/googlefonts/colr-gradients-spec/issues/199 for some prior discussion. Worth contemplating as part of either a COLRv2 or the so-called "boring expansion" (https://github.com/be-fonts/boring-expansion-spec).

[ctrlcctrlv]

It may sometimes even be desired not to union…so perhaps bit 3 can mean "you may need to support doing unions as well, and if you support them, they are requested"

[ctrlcctrlv]

per @Lorp, we also need to put somewhere in OT spec what the fallback behavior is…looping in @peterconstable…even I agree the fallback behavior should be the most common behavior now, which is not my preferred behavior (sigh)

[Lorp]

It seems likely that some fonts will want different behaviour per glyph, such as a font that includes handwriting style glyphs and a flag or logo. Therefore each glyph really needs its own set of control bits.

Btw what happens if combinable glyphs have different colours per layer?

[PeterConstable]

This is stepping into a very non-trivial requirements minefield.

• For single-layer overlaps, control what compositing mode and what blending mode should be used.
• For colour glyphs with multiple layers, how should the layers of one glyph interleave with the layers of the adjacent glyph?
• Do the above need to be controlled contextually?

[ctrlcctrlv]

I don't see a way around addressing this. The current ambiguity is a big problem.

[ctrlcctrlv]

@Lorp I can imagine a “short XOLR” and “long XOLR”.

Short XOLR is as described above.

Long is the typical OT record-based approach—

XolrRecord

    > # big endian
    left_gid:    L
    right_gid:   L
    flags:       L

XOLR

XOLR_FORMAT = """ 
    > # big endian
    version:     L
    flags:       L # if bit 8, then XOLR is long. otherwise, bits 1-7 apply to whole font
"""

[PeterConstable]

we also need to put somewhere in OT spec what the fallback behavior is… even I agree the fallback behavior should be the most common behavior now...

Note that there are existing fonts (certainly for OT-SVG) that expect overlapping glyphs to use simple alpha blending.

[rsheeter]

Vigorously agree with both "This is stepping into a very non-trivial requirements minefield." and "The current ambiguity is a big problem."

I like the general idea of saying what currently happens is the default and defining a way to define modes, including room for more, and starting with some that are only modest steps from current behavior. I worry that I may not adequately appreciate how appalling such a change is for rasterizers.

[PeterConstable]

Wrt current ambiguity, and without adding more in the way of means of controlling, I think a small spec change could be to say something along the lines of

If adjacent color glyphs overlap, simple alpha compositing should be used by default. Applications may also provide alternate, user-selectable compositing or blending modes for this inter-glyph interaction.

[ctrlcctrlv]

I'd have to disagree with that change and would prefer ambiguity to that. It reads as "the current output is correct (dubious), and all other output, including output that would make cursive color font possible, is discretionary".

[Lorp]

Note that considering “adjacency” or “left glyph/right glyph” is insufficient. All glyphs in the same rendering context need to be considered for the possibility of overlaps, for some (as yet undefined) definition of “rendering context”.

[PeterConstable]

Short of introducing new control mechanisms, the only alternatives to the current ambiguity are

1. maintain current ambiguity, meaning there is no way for any colour font to predict whether text will display as the font developer intended in different environments

2. Recommend simple alpha blending by default, with the option for other discretionary behaviour set at runtime by the application.

3. Recommend some other mode by default, with discretionary alternatives set at runtime.

# 2 doesn't help overlap in cursive fonts, but it does maintain compatibility with other existing OT-SVG fonts, which we hope will get reimplemented as COLR.

Re # 3 We'd have to decide just what is that other mode recommended by default. For cursive fonts like the example above, different compositing modes could produce the desired result: source over, destination over, source in, destination in, sourc atop, destination atop. But some things to note about that example are (i) within the area of the intersection of the bounding boxes, the two overlapping strokes cover the same area; and (ii) both have the same flat-color fill. I could imagine glyphs with different types of fill where the choice of compositing mode would matter to the font developer.

Given open questions regarding 3 and a broader concern with back compat, I'm inclined to say 3 isn't a good option.

[PeterConstable]

Note that considering “adjacency” or “left glyph/right glyph” is insufficient. All glyphs in the same rendering context need to be considered for the possibility of overlaps, for some (as yet undefined) definition of “rendering context”.

Could you elaborate a bit?

Suppose a web page has two different containers with text using the same colour font, and the two containers are overlapped, I don't think it would be reasonable to say that data in the font could control how text in the two containers are composited.

But that might not be representative of what you have in mind.

(Tangent: It would also not be reasonable to suppose that one colour font could have data controlling how glyphs are composited if overlapped by glyphs from a different colour font, beyond the alpha that affects how it would be composited with the background in general.)

[Lorp]

Suppose a web page has two different containers with text using the same colour font, and the two containers are overlapped, I don't think it would be reasonable to say that data in the font could control how text in the two containers are composited.

Agreed. What I’m saying is that “text container” is a concept from applications, not rasterizers or shaping engines.

(Tangent: It would also not be reasonable to suppose that one colour font could have data controlling how glyphs are composited if overlapped by glyphs from a different colour font, beyond the alpha that affects how it would be composited with the background in general.)

Agreed too, I think. Good point to raise.

[ctrlcctrlv]

This problem is at the border of shaping (multiple glyphs) and rasterization (single glyphs).

There needs to be a standardized set of modes to know how to composite glyphs from within the same font and the same «text container».

As well as a defined fallback mode.

This is not a purely application level issue any more than shaping is. Color fonts need a way to explain their intended composition across glyphs.

[PeterConstable]

Not sure what you mean by "at the border of shaping". In a text-layout stack, operations are done in a particular sequence, and rasterization/drawing happens after shaping. The only connection between this and shaping is that there is a similarity in that both involve interactions between different glyphs.

There needs to be a standardized set of modes to know how to composite glyphs from within the same font and the same «text container».

At present, given the existing table formats, I think the right answer is that there is a single mode: simple alpha blending. I'm not opposed to possible enhancements for other modes in the future, but that is going to take some time. For this enhancement, I'd be particularly concerned about what is going to be feasible for layout implementations, and so I think it will be important to have engagement from multiple platform or app vendors on this.

As well as a defined fallback mode

As assume that's only relevant if and when there is more than one specified mode.