glyf OVERLAP flags in latest FreeType

[anthrotype]

I the latest FreeType 2.10.x, the (formerly-private, Apple-only) TrueType OVERLAP_SIMPLE and OVERLAP_COMPOUND flags (originally meant to fix winding issues on self-overlapping glyphs for some old Apple renderers) are now being used to enable 4x4 oversampling to mitigate the effects of increased pixel coverage where paths overlap inside a glyph. The flags are being read not only for static but also for variable fonts:

See https://savannah.nongnu.org/bugs/?59026 and my summary in https://github.com/googlefonts/ufo2ft/issues/402#issuecomment-688925786

I argued in the other thread that freetype developers' decision of interpreting those legacy flags for the purpose of enabling the oversampling was a bit unilateral, especially for variable fonts where overlaps are the norm; and also considering that Apple's and Microsoft's font renderers somehow manage to avoid the glitchy rendering of overlapping contours without requiring those flags.

But as things stand, users are already starting to resort to ad-hoc solutions like setting the flags indiscriminately on all glyphs of a VF as a post-process build step (e.g. https://github.com/microsoft/cascadia-code/issues/350).

I am thinking of automatically detecting overlaps in ufo2ft (by testing contour and component intersections via skia-pathops) before building the TrueType glyphs and setting the flags accordingly in each master TTFs. In varLib the default master TTF is the basis for the VF, so if a default master's glyphs have the overlap flag set, the VF will also inherit those. However, the overlap may occur only in some regions of the variation space. That is, a glyph in the default master may not need the overlap flag but some other master will have it set. In this case, fontTools.varLib should check for each glyph whether any master TTF has the overlap flags, and if so set them on the default glyph in VF glyf table.

[apodtele]

First, it has to be recognized that font rasterization is by far the most computationally expensive part of font handling, much more CPU or GPU consuming than VF interpolation, hinting, and the rest combined. Therefore, all issues with rasterization have to be treated with utmost respect. Two approaches to calculate factional pixel coverage for anti-aliased rendering seems to exist: the bilevel oversampling at higher resolution and the curve integration in each pixel. The former is allegedly implemented by Microsoft, the latter is allegedly implemented by others including Adobe and Apple and certainly in FreeType. The oversampling is likely immune to the inflated overlap coverage issue, but the integration is faster and the others likely insisted on the OVERLAP flags to be resurrected with quite strong language to paraphrase: either remove overlaps or set the flags is static fonts. I am just reading the specifications between the lines, I have zero knowledge of actual negotiations. The strong language in static fonts suggests that any mitigation procedures are quite expensive: in FreeType it quadruples the rendering time.

Perhaps, due to backwards compatibility the flags are optional in variable fonts, but I would not be surprised if they become optional but recommended. I assume the others just enable the mitigation strategies for all variable fonts regardless if they have overlaps or not. I would love to benchmark variable vs static font rendering in Core Text. To me that feels like an overkill and, at Werner's suggestion, I enabled the gray oversampling only if the flags are set. The artifacts are rather minor to pay such heavy price. I would rather shift the decision on the font designers to make a call. It is undeniable that detecting overlaps is better done once than every time.

[anthrotype]

I just remembered that when compressing to WOFF2 format, the glyf table's preprocessing transformation does not keep the OVERLAP_SIMPLE/OVERLAP_COMPOUND flags:

https://github.com/google/woff2/issues/123#issuecomment-500371866

so one would have to choose whether to get a smaller webfont or a bigger one (with glyf transformation disabled) that renders without glitches in a major implementation such as FreeType (or even do without overlaps altogether in the design itself, which is not always achievable for VFs)

[anthrotype]

fontTools.varLib should check for each glyph whether any master TTF has the overlap flags, and if so set them on the default glyph in VF glyf table.

@rsheeter made me realize that this algorithm is wrong, because it may well be that none of a glyph's invidual masters require the overlap flags to be set, while at the same time as result of applying the VF deltas a given instance may end up with self-overlapping contours or components in some regions of the variation space.. I don't see a way to automatically detect this in all circumstances.

I guess the font designer will have to explicitly set those flags in the neutral master (when ufo2ft implements the proposed new glif.lib key), and these will be automatically inherited by the variable font.

[apodtele]

@anthrotype I understand that explicitly flagging overlaps might not be possible in some font formats. Do you suggest that FreeType should then care to detect the overlaps, remove them or mitigate potential artifacts? This is too much to expect from a low level library. To retain explicit OVERLAP_SIMPLE or OVERLAP_COMPOUND, we respectfully set FT_OUTLINE_OVERLAP and mitigate artifacts. Our clients can do so manually for a good reason or no reason at all, having been warned about the performance hit in the documentation.

FreeType only does what user asks or font communicates. Does it still sound unilateral to you?

[apodtele]

@anthrotype said

(originally meant to fix winding issues on self-overlapping glyphs for some old Apple renderers)

There was never any confusion about TrueType non-zero winding rule: both Apple and Microsoft specifications agree on this. It is Type1/CFF that specified even-odd winding and switched to non-zero winding for variation CFF2.

It is bizarre that these flags are associated with the winding rule. They were meant simply to warn about overlaps. PostScript printer drivers (not Apple renderers) should be careful with the winding rule. Now FreeType should be careful about the edge artifacts.

@Lorp

[Lorp]

It seems to me that the spec has been 100% clear on this since the early 1990s, and therefore I don’t agree with any suggestions to bypass it. To avoid the a bug in a super optimized renderer, a flag to switch to a slow renderer can be regarded as a hack, or more charitably as a cached value to say “don’t even try”. So (without the flag) FreeType would ideally detect overlaps during a fast render, abort the render, and re-render using the slow renderer, and internally flag that glyph in case it renders it again at a different size or variation.

[apodtele]

@Lorp Actually it is the other way around. Since early 90s the intersecting contours were discouraged because some implementations used wrong winding rule. To alert them, these flags were used when necessary. It is the variation fonts that broke all rules quite recently but forgot about the flags. Clearly, if overlaps are so important for variation fonts, the very least you can do is to warn about them.

FreeType has no problem with traditional fonts. It took me a week to find overlaps in traditional fonts: I found the Euro symbol on Mac OS 9 where it is not even visible because of hinting. With hinting out of scope in variation fonts, you still refuse to flag overlaps. Wow, just wow.

Detecting overlaps or removing them is too hard to be viable option for FreeType.

[justvanrossum]

FWIW I agree with Laurence. It's 2020, in VF overlaps are the norm, not the exception. It's pretty crazy that all fonts would have to provide information so one specific renderer can work around its own limitations.

[apodtele]

@justvanrossum Don't misrepresent what I said. I did not ask to make the flags required (the ship has sailed). Nothing stops you from using them going forward though. It may or may not be a good idea depending on the intended use case and how visible the artifacts are. This option must be afforded to the font developer.

[Lorp]

@apodtele Have you checked the Times New Roman glyphs Ccedilla and ccedilla in FreeType? These have used overlapping composites forever, yet the OVERLAP_COMPOUND bit is not set in the version preinstalled on the Mac I am typing on. But perhaps composites are not problematic with overlaps.

[apodtele]

@Lorp What an excellent case in point! The overlap in Ccedilla and ccedilla of Times New Roman is removed by hinting so that the contours barely touch each other. There is not need to flag them because neither the winding rule, nor the coverage issue matters. The font designer made a good decision.

[twardoch]

Those flags were private Apple flags that were not even part of the Apple TrueType spec. I only discovered them in a “footnote” (well, appendix) on the Apple site. We incorporated them into OTVar but since they were “reserved” before and we did not change the glyf table version, suddenly making them required would have been a bit hardcore. WOFF2 was made under the assumption that these flags are reserved, hence unused, and optimized them away.

[twardoch]

Here’s the original issue.

Apple uses a faster rasterizer which uses the even-odd rule and a slower one that uses the winding number rule. The faster Apple rasterizer does not have the problem when the intersection happens once but it does show holes if the intersection happens twice:

https://github.com/twardoch/test-fonts/tree/master/varia/160413-EvenOddTT

They use the flag to force the slower rasterizer.

[apodtele]

@twardoch They were overlooked, but they were always public. Here is an archived link from 1998. Enjoy!

Curiously, the flags were first disclosed in relation to TrueType GX and now I am arguing about them with the variation crew.

[twardoch]

That's not the point, the point is that the actual TrueType spec for them said (and still says) "reserved, set to zero". https://developer.apple.com/fonts/TrueType-Reference-Manual/RM06/Chap6glyf.html

OpenType also said of them "reserved, set to zero".

The fact that Adobe rasterizers for years executed opcodes in the CFF table that in fact were defined as reserved in the CFF spec does not make those opcodes "legitimate". Some of these opcodes are actually documented in OpenType pre-1.25, yet still.

[apodtele]

My point is simple: OVERLAP_SIMPLE and OVERLAP_COMPOUND are useful flags.

They might be unsupported in WOFF2 or CFF, or misunderstood as the winding rule flags, or ambiguous between different variation instances, etc, etc. This has no bearing, they are still a USEFUL warning for actual rasterizer implementations and the specifications for static fonts state:

• either set this flag in the derived glyph data, or else should merge contours to remove overlap

and the variation fonts might optionally do that as well.

[Lorp]

@apodtele so FreeType fails on Times New Roman in non-hinting environments or when the font size is large?

[apodtele]

@Lorp Literally, two (2) pixels where the contours intersect might have wrong coverage values if hinting is disabled for Ccedilla and ccedilla of Times New Roman. The worst case scenario is when contours superimpose.

[Gr3gH]

Werner asked if I would make a few comments about the Microsoft TrueType rasterizer and the OVERLAP_SIMPLE and OVERLAP_COMPOUND flags.

Microsoft received their last drop of TrueType from Apple in 1992 which did not include any of their GX code. The Windows 3.1 TrueType rasterizer and the TrueType specification did not include those two flags.

In 1992, Microsoft started on a major rewrite of the TrueType code. Some of the algorithms we kept while others were redesigned. Significant work was done in the scan conversion component (which is relevant here) — although many parts of the algorithm were at least similar to what Apple had provided, but we explicitly did not add support for these flags (which we discovered after we had shipped with Windows 95).

Our basic fill algorithm is non-zero winding based on a scanline fill. For each scanline we calculate intersections. (The scanline is determined by the pixel center in “y”). An intersection is either an on-transition or an off-transition (see: Apple TrueType Spec. We sort our on-transitions and our off-transitions (so we go left to right in increasing x). Each “on” is paired with an “off”, and we fill between the “on” and “off” transition. Overlaps are handled because the “on” and “off”-transitions will overlap, creating essentially a double fill.

We do not handle the case of overlaps when the “background” is filled, such in dark circle glyphs, e.g. “black circled number”. There is discussion on changing this, but so far, we’ve not determined a positive risk vs. benefit for a core change to the scan converter. “White on Black” VF Rasterization Issue GregH

[anthrotype]

Thank you @Gr3gH for your comments about MS TrueType rasterizer. However, the issue at hand here is not the fill algorithm (FreeType also uses nonzero winding rule), but with graphical glitches that show up where outlines self-intersect or intersect one another as result of incresed pixel coverage. Basically the pixels at the edges of overlapping contours appear darker. The screenshot below is Cascadia Code font (I got it from the linked freetype issue):

I wonder whether/how MS rasterizer deal with these, as in my testing it doesn't show such rendering glitches.

[apodtele]

My educated guess is that MS oversamples and averages the mono rasterizer to achieve anti-aliasing. Using oversampling (with associated costs) is the key to dealing with overlap glitching.

Let's not pick on FreeType algorithm either. It is now confirmed that macOS uses similar rasterizer. FreeType would like to be warned about overlaps do fall back on oversampling (with associated costs) to deal with overlap glitching.