[otfautohint] Mishinting of glyph?

[madig]

I'm trying to use the new otfautohint on https://gitlab.gnome.org/GNOME/cantarell-fonts and there's something weird going on with Ef-cy:

1. The hints seem to just barely miss the lower belly in the Regular:
2. The bottom point of the belly seems to be ripped apart by hinting, making the glyph all wonky:

Here's the dump of the glyph:

$ tx -g uni0424 -dcf build/src/Cantarell-VF.otf
### CharStrings (flattened)
--- glyph[tag]={name,path}
[520]={uni0424,
  69.51 72.78 411.59 72.71 -72.71 160.13 16.58 421.66 -53.81 76.54 -430.56 -559.1 466.15 75.86 -466.15 -75.86 467.79 60.89 6 blend
  hstemhm
  43 85 230 81 231 85 20 -19 -63 85 72 -98 -60 89 74 -99 -63 85 6 blend
  hintmask[5C]
  358 29 -32 1 blend
  -20 rmoveto
  81 -60 89 1 blend
  hlineto
  hintmask[3C]
  734 -81 60 -89 1 blend
  vlineto
  35 -646 -24 46 18 -20 2 blend
  rmoveto
  hintmask[9C]
  228 134 105 177 175 -131 103 -222 -16 18 -9 11 -7 8 -10 13 -9 10 7 -11 5 7 11 -18 8 blend
  hvcurveto
  hintmask[5C]
  -226 -133 -106 -179 -173 130 -102 220 14 -20 8 -13 -3 -6 12 -11 8 -13 -6 12 4 -8 -9 21 8 blend
  hvcurveto
  9 73 -6 1 -54 72 2 blend
  rmoveto
  hintmask[9C]
  -172 -102 78 131 129 98 76 166 -27 32 -15 19 13 -19 24 -32 25 -35 18 -19 25 -20 31 -33 8 blend
  hvcurveto
  hintmask[5C]
  175 103 -79 -133 -127 -100 -75 -168 23 -35 14 -21 -23 20 -22 34 -26 33 -16 22 -16 19 -28 35 8 blend
  hvcurveto
}

The font file, if you don't want to compile it yourself: Cantarell-VF.zip

Is this a user error and if so, what should I do?

[skef]

Hinting with otfautohint (or psautohint) should never change the path, so if the path itself is different then it's a bug in the program.

[skef]

I've taken a deeper look into this.

The change in hinting behavior here is intentional. otfautohint attempts to hint glyphs in variable fonts in the way it would if overlap were removed. In a static version of this glyph there would be no "bowl", and the curves to the left and right of the lower curved section are apparently too steep (given the various heuristics in the program) to cause it to add a stem there.

As it happens, this particular glyph, at that ppm, is rendered better (in FreeType, at least) when there is a stem there. We have found that the updated approaches and bug fixes in otfautohint do better than those in psautohint generally, but naturally there are specific glyphs that will look worse. Hinting, and the application of hints when rendering, is more of an art than a science.

So I am afraid our advice is that this result is consistent with the updated approach in otfautohint of simulated overlap removal, and if the program has a result you don't like for a particular glyph, the fallback (as before) is to hint manually.

[madig]

Thanks for looking into this. What tool can I use to hint stuff manually, though?

[skef]

@madig that's a good question. Tooling for manual PostScript-style hinting in general has deteriorated a bit, and there hasn't been new work supporting the variable case specifically (yet?).

There are two problems of manual hinting and one is relatively straightforward: picking the stem regions. That problem amounts to adding corresponding stem regions to each master and making sure they don't change order between masters.

If the regions never get too close to one another in any point in design space, you're done. You can basically just add those regions to each of the masters and the normal mechanisms used to build fonts will combine them. If they do overlap one needs hint masks/hint substitution, and that gets trickier, as that is normally "done for you" by tools and in this case the different mask/substitution sets need to be combined into a single set that works across all masters. I'm afraid I don't have good advice about this for now beyond possibly just inspecting the different substitution patterns output by tools in the UFO glif sources and reconciling them at that level.

[vitorsr]

It is unclear to me what the option’s behavior is when hinting CFF2 fonts but would allow_changes have any effect here?

Plus, are less-used Private DICT Operators such as StemSnapH taken into account and would that also have any effect here?

[madig]

@skef: so... I ran otfautohint git main over the source Ef-cy (because v4.0.0 can't deal with hyphens in glyph names) and now have some hints in a glif lib key. How do I stuff them into the actual OTF made with ufo2ft/varLib? Compile with makeotf and transplant the CFF2 entry for Ef-cy?

[skef]

@madig There are different directions you could go here, depending on how temporary or permanent you want the fix to be. My approach would probably be to hint the UFOs as a group of masters using the relevant flags of otfautohint. That should have more or less the same effect as hinting the built file. Then I would make any tweaks to Ef-cy that are needed by playing with the hinting layer. Then I would build with buildmasterotfs (or makeotf directly) and bulidcff2vf .

@vitorsr I believe allow_changes has no effect for CFF2 because fiddling with path data for VF masters is more trouble than it's worth. Whereas a double point (no position change) has no purpose in a glyph in a static font, there are all sorts of reasons one might want that in one master in a VF. More generally I eliminated some of the potential changes for that flag in the python port, not because they're not valid but because in contemporary font development a hinting stage is a strange place to be imposing them -- one should really use some sort of sanity or validity checker for that sort of thing.

The hinter does digest and make use of the other parameters like StemSnapV, but the effects of those tend to be subtle at best. If you can get a better result from just tweaking the stem size parameters, that's mostly the result of luck.