should transformed components be decomposed in TTFs?

[anthrotype]

When generating TTFs, Glyphs.app decomposes all components which have any transforms (scale, flip, rotation, etc.) applied to them.

@schriftgestalt, is this another legacy compatibility thing? Any custom parameter to disable it?

If this is confirmed, we should find a way to handle this betweem fontmake, glyphsLib and ufo2ft.

Maybe we could add an option in ufo2ft outline compiler, and have fontmake enable it when generating TTFs from .glyphs sources.

I wonder, is this a Glyphs.app-only thing, or should it be generalized or even made default in ufo2ft regardless of the glyphs or UFO source input?

What do other font editors do in this case?

/cc @schriftgestalt @twardoch @behdad

[schriftgestalt]

The components are decomposed because it in most cases would disturb the hinting that is attached to the base glyph. I can add a parameter to prevent that if that would be needed.

[twardoch]

I believe that ttfautohint’s --composites option should — at least in theory — work fine with transformed components, because it disables TT instructions inherited from the parent glyphs and applies instructions on the outlines of the composite glyphs. When Werner originally developed this, it was the only, default behavior of ttfautohint. Back then, I asked him to make this behavior optional because the separate instructing of all composite glyphs significantly increased the file size, potentially offering little benefit in many fonts. So now by default, if you don’t use --composites, ttfautohint will not apply any hinting to the composite glyphs, relying solely on the hinting inherited from the parent glyphs.

But the case of transformed (I mean transformed in a way other than shifting, of course) components is an interesting one: instructions inherited from the parent glyphs will most likely be useless in that case. It would be useful if Werner added an option, which does perform full hinting on glyphs which have transformed components, but not do the separate hinting on untransformed components. That should actually be the default behavior, because that would offer the best balance between effectiveness and size.

[twardoch]

BTW, I think having a custom parameter in Glyphs that disables the decomposition of transformed components would be useful. After all, the current behavior defeats the purpose of TrueType’s support for transformed components. Glyphs’ default behavior may stay as it is, but if someone is post-processing the output, or building variable fonts, it would be useful to keep transformed components.

[anthrotype]

Ok, so I'll add an option in ufo2ft to decompose such transformed components (anything other than shifting).

I think fontmake should pass that option by default when generating from .glyphs source to emulate the current Glyphs.app behaviour.

Then if @schriftgestalt defines a new custom parameter, we can add support for that in glyphsLib, and fontmake can read that to decide whether to decompose transformed components or not when generating from .glyphs sources.

fontmake script could also have a command-line option, e.g. --decompose-transformed-components, which overrides such format-specific default behavior and enables or disables the feature.

[brawer]

command-line option

<mantra>A zoo of configuration options is difficult to understand. The build process would be less complicated (and easier to reproduce) if the output was completely determined by the input, without side channels.</mantra>

[behdad]

Agree with not adding options for everything...

So, it occurs to me that if we fix ttfautohint to only hint composites if any component has a non-translation transform, then it's good to never decompose components. Georg, do you agree? We should get this on Werner's radar.

[schriftgestalt]

I agree. If ttfautohint would work like this, I could not decompose components if autohinting is applied. But adding an parameter would be a good idea, too.

[schriftgestalt]

Added Keep Transformed Components (like the existing Keep Overlapping Components).

[anthrotype]

There is another reason, besides hinting, why one would want to decompose transformed components, which is the issue reported in https://github.com/googlei18n/fontmake/issues/22, namely the fact that different rasterizers behave differently as regards composite glyphs and transforms that reverse the winding direction, which manifests itself when similar components overlap with each other.

The consesus over there seems to be to always decompose such composite glyphs, to prevent things like that.

I think what we should do is to always decompose components when they (1) intersect other contours/components, and (2) have a reversing transform.

Support for the custom parameters that Georg mentioned could also be added to fontmake/glyphsLib/ufo2ft, but if ttfautohint does the right thing (i.e. only hint composites if any component has a non-translation transform, like Adam and Behdad suggested) then we wouldn't need an extra custom parameter that optionally decomposes them.

[behdad]

There is another reason, besides hinting, why one would want to decompose transformed components, which is the issue reported in notofonts/noto-fonts#22, namely the fact that different rasterizers behave differently as regards composite glyphs and transforms that reverse the winding direction, which manifests itself when similar components overlap with each other.

Not so fast! First, we have not verified yet that rasterizers have different behaviors in this case. Second, even when decomposing, the overlap causes a problem, so, we might need to flip contour direction during decomposition for mirrored contours, but that will screw up point numbers, if we care about those ever.

The consesus over there seems to be to always decompose such composite glyphs, to prevent things like that.

Not quite sure yet. We first need to gather data points re different rasterizer behaviors. A test font is a first step.

I think what we should do is to always decompose components when they (1) intersect other contours/components,

That's a rather expensive thing to detect,

and (2) have a reversing transform.

Would be cleaner if we can avoid this...

Support for the custom parameters that Georg mentioned could also be added to fontmake/glyphsLib/ufo2ft, but if ttfautohint does the right thing (i.e. only hint composites if any component has a non-translation transform, like Adam and Behdad suggested) then we wouldn't need an extra custom parameter that optionally decomposes them.

Right. Let's see if we can convince @lemzwerg to do this...

[lemzwerg]

In general I don't think that this suggestion will work satisfactorily with ttfautohint. The idea of using the components' hints only by default is based on the 'base glyph plus diacritics' model where the components don't interact with the base glyph of the composite. In particular, you get only satisfactory results if the ROUND_XY_TO_GRID composite flag is set – if you are going to transform components non-trivially, I suppose this flag is normally not set (since you can't reliably connect glyph shapes of components otherwise) and you have to use ttfautohint's --composite command line option.

Another issue is that --composite makes ttfautohint internally decompose composite glyphs, and suddenly all components of the glyph take part in the hinting algorithm, which can lead to differently positioned stems or composites. Doing this for some glyphs only might thus yield visually bad results.

Mixing up the two 'modes' would make sense, of course. For example, a composite's base glyph might be itself a composite that doesn't use ROUND_XY_TO_GRID. However, this is something that ttfautohint can't handle automatically – how should it know which is which? Theoretically, I could add commands to ttfautohint's control instructions file that enable this feature manually. However, I wonder whether there are enough use cases for it.

[anthrotype]

Note that fontmake (specifically ufo2ft outlineCompiler via fonttools TTGlyphPen) currently always sets the ROUND_XY_TO_GRID flag; and that there is no (public) way in UFO to specify which glyf flags to enable for each components.

[anthrotype]

In https://github.com/googlei18n/fontmake/issues/22, @behdad wrote

this is in fact ambiguity in the OT spec. We discussed it and I asked @twardoch to create a test font so we can test different rasterizers' behavior regarding composite glyphs and reversing transforms.

Any news on this front?

I guess in order to verify this, we should create a test case at https://github.com/unicode-org/text-rendering-tests.

I remember there was a related issue in Noto which had to do with flipped overlapping components, which has been resolved by decomposing them in the source itself: https://github.com/googlei18n/noto-fonts/issues/859#issuecomment-321902539 From the reports and screenshots, it looks like this affected both Windows (8.1 and 10) and FreeType rasterizers (via FontView, fontdiff, hb-view).

[anthrotype]

While I was comparing different font editors' behavior as regards transformed and/or overlapping components in TTFs, I noticed that the latest version of Glyphs.app (2.5b 1071) no longer automatically decomposes transformed composite glyphs on export, unlike I had originally reported here. I also tried ticking the autohint, and the remove overlap checkbox, even tried to manually add hinting instructions to a base glyph to see whether the transformed component would be decomposed as a result, but nothing. Composites seems to be kept all the time (unless of course there are mixed contours/components, in which case they are decomposed).

Therefore I don't understand the purpose of the custom parameter that @schriftgestalt mentioned earlier https://github.com/googlei18n/fontmake/issues/253#issuecomment-317281334, which are supposed to "keep transformed components" and "keep overlapping components". They don't seem to be doing anything, as the components are kept anyway, with or without those parameters. I'm a bit confused (I must be doing something wrong).

On the other hand, I see that RoboFont always decomposes components that have any transform applied and could not find any option to control that.

[anthrotype]

I made a simple font with only two glyphs: "A" contains a simple contour, "B" is a composite glyph that has two components, one same as "A", the other with a xScale of -1.0 (i.e. flipped vertically).

This is how the string "AB" looks in macOS Sierra with FontBook.app:

This is in Windows 10 with Notepad.exe:

This is in Ubuntu 16.04 with LibreOffice:

Here is the quick-and-dirty font that I used: Untitled-Regular.ttf.zip

---

So... I believe this is enough information to conclude that such ambiguous situation should be avoided somehow: I'm talking specifically of a component that has a transform applied, not any transform, but one such as the determinant of the 2 x 2 matrix is negative and hence the path direction is inverted (e.g. flip), and this component overlaps with another contour or component that has not been transformed like that.

The solutions I see is either go through the relatively expensive process of checking whether component overlap with each other, and decompose the ones that have a direction-flipping transform (while also reversing the direction as we decompose them); or we do nothing and we let the designer fix this in the sources.

[anthrotype]

BTW, also the font editors don't agree on the way to show the filled-in preview of that "B" glyph with the overlapping components (one of which is flipped).

Glyphs.app displays it like in the macOS screenshot above, i.e. all black.

Whereas Robofont, TruFont and FontLab Studio display it with a white hole in the middle, similar to the other two screenshots.

[schriftgestalt]

That version 1071 decomposes less is not intentional. I’ll fix that.

[khaledhosny]

Would setting OVERLAP_COMPOUND bit on composite glyph flag make any difference on Apple platform?

[khaledhosny]

(not that I know any tool that allows setting such flags).

[schriftgestalt]

Glyphs displays the flipped components overlap as black as it (normally) will decompose and revert the outline. So you should always get the same result.

[moyogo]

We should have keys like "keep transformed components" and "keep overlapping components" so the user can control what is happening. The default can be the most common case.

[anthrotype]

I understand that Glyphs.app works like that, and that we may wish to replicate this behaviour in fontmake. However, the ambiguity remains as I showed in the above screenshots. Maybe we should raise the issue with the OpenType mailing list.

/cc @behdad @PeterCon @nedley

[schriftgestalt]

When a flipped component is decomposed, the paths are reversed to avoid rendering problems like you see above.

[anthrotype]

@khaledhosny after https://github.com/fonttools/fonttools/commit/704e26b41dfae79a91fe08c1951b000d7d348141, fonttools can now read/write the OVERLAP_COMPOUND flag. I tried to set that on the flipped component but it didn't change anything. Should it be on the first component maybe? I suspect that flag is not the culprit.

[khaledhosny]

I think it should be on the first component, also you might want to try it on normal overlapped components (no flipping) and see if it has any effect there (I’d have tried myself, but my mac does not want to boot anymore).

[anthrotype]

So this issue resurfaced again: https://github.com/googlei18n/noto-fonts/issues/1367

I've shown above with a test font how the different OSes render these: https://github.com/googlei18n/fontmake/issues/253#issuecomment-335535293

I think fontmake needs to decompose these overlapping components with reversing transform. Font designers using Glyphs.app expect this to be done automatically.

The ufo2ft DecomposeComponents filter should check for these and do the right thing.

[anthrotype]

I tried to set that on the flipped component but it didn't change anything. Should it be on the first component maybe?

@khaledhosny I tried again and, indeed, I can confirm that if I set the OVERLAP_COMPOUND flag on the first component of the composite glyph with overlapping paths (one of which was flipped), then it does changes the rendering on macOS (FontBook.app macOS 10.15.6), so that it becomes consistent with Windows and Freetype behavior:

Untitled-Regular_OVERLAP_COMPOUND.ttf.zip

[arrowtype]

Just as a small addition here, I find that it is necessary to decompose both nested and transformed components, in order to get decent results on postscript printers (i.e. basically all printers).

https://github.com/arrowtype/recursive/issues/412