Sparse / Partial Masters and Anchors

[weiweihuanghuang]

I'm having some issues with this VF project with UFOs where I have some sparse UFO masters, I get the following error:

INFO:fontTools.varLib:Merging OpenType Layout tables
Traceback (most recent call last):
  File "/usr/local/bin/fontmake", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python2.7/site-packages/fontmake/__main__.py", line 407, in main
    project.run_from_designspace(designspace_path, **args)
  File "/usr/local/lib/python2.7/site-packages/fontmake/font_project.py", line 914, in run_from_designspace
    **kwargs
  File "/usr/local/lib/python2.7/site-packages/fontmake/font_project.py", line 969, in _run_from_designspace_interpolatable
    designspace, output_path=output_path, output_dir=output_dir, **kwargs
  File "/usr/local/lib/python2.7/site-packages/fontmake/font_project.py", line 411, in build_variable_font
    inplace=True,
  File "/usr/local/lib/python2.7/site-packages/ufo2ft/__init__.py", line 577, in compileVariableTTF
    ttfDesignSpace, exclude=excludeVariationTables, optimize=optimizeGvar
  File "/usr/local/lib/python2.7/site-packages/fontTools/varLib/__init__.py", line 916, in build
    _merge_OTL(vf, model, master_fonts, axisTags)
  File "/usr/local/lib/python2.7/site-packages/fontTools/varLib/__init__.py", line 688, in _merge_OTL
    merger.mergeTables(font, master_fonts, ['GSUB', 'GDEF', 'GPOS'])
  File "/usr/local/lib/python2.7/site-packages/fontTools/varLib/merger.py", line 111, in mergeTables
    for m in master_ttfs])
  File "/usr/local/lib/python2.7/site-packages/fontTools/varLib/merger.py", line 1003, in mergeThings
    super(VariationMerger, self).mergeThings(out, lst)
  File "/usr/local/lib/python2.7/site-packages/fontTools/varLib/merger.py", line 97, in mergeThings
    self.mergeObjects(out, lst)
  File "/usr/local/lib/python2.7/site-packages/fontTools/varLib/merger.py", line 76, in mergeObjects
    mergerFunc(self, value, values)
  File "/usr/local/lib/python2.7/site-packages/fontTools/varLib/merger.py", line 1003, in mergeThings
    super(VariationMerger, self).mergeThings(out, lst)
  File "/usr/local/lib/python2.7/site-packages/fontTools/varLib/merger.py", line 97, in mergeThings
    self.mergeObjects(out, lst)
  File "/usr/local/lib/python2.7/site-packages/fontTools/varLib/merger.py", line 76, in mergeObjects
    mergerFunc(self, value, values)
  File "/usr/local/lib/python2.7/site-packages/fontTools/varLib/merger.py", line 1003, in mergeThings
    super(VariationMerger, self).mergeThings(out, lst)
  File "/usr/local/lib/python2.7/site-packages/fontTools/varLib/merger.py", line 97, in mergeThings
    self.mergeObjects(out, lst)
  File "/usr/local/lib/python2.7/site-packages/fontTools/varLib/merger.py", line 76, in mergeObjects
    mergerFunc(self, value, values)
  File "/usr/local/lib/python2.7/site-packages/fontTools/varLib/merger.py", line 1003, in mergeThings
    super(VariationMerger, self).mergeThings(out, lst)
  File "/usr/local/lib/python2.7/site-packages/fontTools/varLib/merger.py", line 99, in mergeThings
    self.mergeLists(out, lst)
  File "/usr/local/lib/python2.7/site-packages/fontTools/varLib/merger.py", line 82, in mergeLists
    assert allEqualTo(out, lst, len), (len(out), [len(v) for v in lst])
AssertionError: ((3, [3, 3, 3, 3, 1, 1, 2, 2]), 'list', '.Coverage', 'MarkGlyphSetsDef', '.MarkGlyphSetsDef', 'GDEF', '.table', 'table_G_D_E_F_')

I understand issue is to do with the anchors, and the issue doesn't present when I don't have the sparse masters.

Is there something I need to do especially with the anchors when I have sparse masters – do the sparse masters all need to have the same glyphs or something. I've seen that the order needs to be consistent across masters but I'm not sure what to do with a sparse master

I know the glyphs in the sparse masters are interpolable as the they build fine with Glyphs App.

[madig]

By sparse UFO master you mean an actual UFO with less glyphs than the others? Or do you mean Glyphs-style brace layers?

[anthrotype]

If you could share (publicly or privately) the sources it could help with debugging. Please if you can try not to use python 2.7 as it's no longer supported.

[arrowtype]

As far as I understand it (though I could definitely be incorrect / behind), "sparse masters" in the typical sense are not currently supported by the UFO/designspace/FontMake workflow.

You can, however, point to "support" layers as sources. Mutator Sans does this:

    <source filename="MutatorSansLightCondensed.ufo" familyname="MutatorMathTest" stylename="LightCondensed" layer="support.crossbar">
      <location>
        <dimension name="width" xvalue="0"/>
        <dimension name="weight" xvalue="700"/>
      </location>
    </source>
    <source filename="MutatorSansLightCondensed.ufo" familyname="MutatorMathTest" stylename="LightCondensed" layer="support.S.wide">
      <location>
        <dimension name="width" xvalue="1000"/>
        <dimension name="weight" xvalue="700"/>
      </location>
    </source>
    <source filename="MutatorSansLightCondensed.ufo" familyname="MutatorMathTest" stylename="LightCondensed" layer="support.S.middle">
      <location>
        <dimension name="width" xvalue="569.078000"/>
        <dimension name="weight" xvalue="700"/>
      </location>
    </source>

https://github.com/LettError/mutatorSans/blob/5c179fe3ba1cb39b7922389eb942e7beb9614742/MutatorSans.designspace#L51-L68

I've tried this workflow and got it working well enough, but found that it was a fairly big additional obstacle in getting consistency-checking tools and scripts to work (from Prepolator to my own little drawing-QA scripts).

The way I tried it:

• The contrast of the Recursive Mono w is a bit much around weight 500. So, I used Skateboard to export the necessary 500-weight "Preview" masters (4, in my case, for Casual and Slant axes).
• I copied the ws from those Preview master into a new layer called something like support.w.layer in my ExtraBold UFOs. I adjusted these drawings to have less stroke contrast.
• I then included these support layers as sources in my designspace. Because I kept those support layers otherwise empty, this "fixed" the w without affecting anything else.

However, the amount of obstacles this gave me in checking/prepping masters for compatibility was ultimately not worth it. This is a feature I may go back and add in later, but was adding so much additional complexity, I didn't find it worthwhile to maintain throughout my entire design cycle.

Maybe, however, you could solve this by using of a sparse-master approach, but having a build step that turns things into support layers.

Of course, it would be really nice if FontMake did eventually support sparse masters!

[anthrotype]

Well, support layers are sparse masters. What exactly do you mean by that?

[anthrotype]

Ok I see the problem. Ufo2ft assumes that when a source is a whole UFO then it can have features of its own and makes kern and mark features for it (depending on the presence of kerning and anchors), which end up in a GPOS and GDEF table; whereas it doesn't do that for sources that are specified as support layers within a UFO. There is a mechanism for disabling the generation of automatic kern/mark features for a specific UFO, by setting some lib key in the lib plist. Maybe that could be used to prevent ufo2ft from generating incompatible GPOS or GDEF tables when a UFO is meant to be treated as a sparse master.

[anthrotype]

What is most typical workflow when designing fonts with sparse masters, to make separate master fonts or separate support layers? I was under the impression that the latter would be more common in the UFO world, as the example of MutatorSans above suggests.

[arrowtype]

What is most typical workflow when designing fonts with sparse masters

I can't answer this in a general sense, but in my experience, tools tend to work best when glyphs are all on the same layer, in separate masters.

As a simple example: it's easy to write a Python script for RoboFont to loop through AllFonts(), and check for similarity between points, anchors, unicodes, etc. It adds significant complexity to have to also check support layers in such a script.

As another example, Prepolator doesn't currently show support layers, even if they exist in a designspace which I open via Prepolator. I assume this is for similar reasons to my own scripts: it's more complex to check for and open support layers.

[anthrotype]

there's another difference between DS sources specified as whole UFOs vs support layers. For the latter, we only compile tables relating to glyph outlines and glyph metrics, see:

https://github.com/googlefonts/ufo2ft/blob/33954355c9c00c59e3b20645444f865ccd77e907/Lib/ufo2ft/constants.py#L3-L6

whereas the whole UFO sort of sources are treated as being "non-sparse" and as such we build also vertical font metrics tables and font-wide metadata and names.

So if one want to use a distinct UFO source as a sparse master (instead of a support layer), one would then also have to duplicate all these font-wide info from the neutral master if they do not want them to participate in, e.g. defining the MVAR table. Leaving these info blank would most likely end up generating different hhea and OS/2 tables for the sparse masters.

[justvanrossum]

Ufo2ft assumes that when a source is a whole UFO then it can have features of its own and makes kern and mark features for it (depending on the presence of kerning and anchors), which end up in a GPOS and GDEF table; whereas it doesn't do that for sources that are specified as support layers within a UFO.

I ran into this a while ago but didn't have the opportunity to dig in deeper and report. I think this is a misguided assumption. If I want to use a specific (non-default) layer as a master, I expect this to work identically as when I would want the default layer. Vice versa, I'd expect a "whole ufo" (default layer) to work as a sparse master as well.

[justvanrossum]

Perhaps it needs to be specified explicitly whether a master is sparse or not.

[madig]

Source Sans Pro/Variable uses standalone UFOs as sparse masters last time I checked.

[weiweihuanghuang]

After having worked on both .glyphs with brace layer and UFO with entire masters for interpolation correction, I personally prefer working with entire masters actually – it feels much more robust to be able to see the whole master.

[weiweihuanghuang]

So in the meantime to get my setup working, I need to copy all those sparse masters to one of the main UFOs as a support layer?

Or, what does Source Sans Pro do to make it work?

[justvanrossum]

Or, what does Source Sans Pro do to make it work?

Wild guess: perhaps it doesn't use mark/base anchors on glyphs that may be sparse.

[weiweihuanghuang]

I wonder if it will work if I have the full character set in the sparse masters, but leave the unused glyphs empty and set a mute in the .designspace for those glyphs.

[justvanrossum]

I don't think varLib supports glyph muting.

I wouldn't mind having a look if you can share your UFOs + .designspace files (that cause the initial error) with me privately. (my gh user name @ gmail.com)

[anthrotype]

@weiweihuanghuang Try to add a "com.github.googlei18n.ufo2ft.featureWriters" key to the sparse UFO's lib.plist containing an empty array of feature writers. This will override the ufo2ft's default list of writers (i.e. KernFeatureWriter and MarkFeatureWriter) for that specific UFO. An empty array means "do not generate any automatic features". Since the sparse UFO will not contain any features.fea of its own, the generated master TTFs will also not contain any OTL tables.

To add that lib key from python you can do:

import defcon

font = defcon.Font("YourSparseMaster.ufo")
font.lib["com.github.googlei18n.ufo2ft.featureWriters"] = []
font.save()

The lib.plist would look like this:

<?xml version='1.0' encoding='UTF-8'?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
  <dict>
    <key>com.github.googlei18n.ufo2ft.featureWriters</key>
    <array/>
  </dict>
</plist>

Let me know if that works for you, thanks

[madig]

I remember that we explicitly turn off feature generation for sparse layers in ufo2ft precisely due to the sparseness.

[anthrotype]

we explicitly turn off feature generation for sparse layers in ufo2ft precisely due to the sparseness.

of course, sparse layers can't have their own features.fea. The problem here is that, when one attempts to use a distinct UFO as a sparse master (i.e. its main layer contains only some but not all the glyphs of the neutral master), ufo2ft will assume that the UFO source is not sparse and hence it will try to build kern and mark/mkmk features, possibly leading to incompatible GPOS/GDEF tables that varLib fails to merge.

[anthrotype]

Perhaps it needs to be specified explicitly whether a master is sparse or not.

yes, maybe the DesignSpace source element can have some attribute that make that more explicit. Let's think about that.

[madig]

Which implies that much like sparse layers, sparse masters can not change any features in any way, i.e. you can't move anchor positions, only ever outlines and spacing. Hm. Kerning?

[justvanrossum]

Which implies that much like sparse layers, sparse masters can not change any features in any way, i.e. you can't move anchor positions, only ever outlines and spacing. Hm. Kerning?

Right. I can see sparse kerning (to touch up specific pairs at a specific location) as a potentially useful thing. I'm not sure if the use case is very strong, though. (A master could for example only contain kerning, and not a single glyph.)

Anchor positions is perhaps the bigger issue here. If you want an intermediate glyph outline, you'd want any anchors to also interpolate with it.

Tangential aside: I've worked on a project where we'd have intermediate masters for spacing but not outlines. I achieved it with a gvar post processing script. The use case was possibly a little obscure: it was to achieve perfect metrics compatibility with legacy fonts at specific locations, while still saving space by not having outlines there.

[madig]

The use case was possibly a little obscure: it was to achieve perfect metrics compatibility with legacy fonts at specific locations, while still saving space by not having outlines there.

But useful, because this might apply to any larger project that needs to be made variable :) Good idea.

[weiweihuanghuang]

Since the sparse UFO will not contain any features.fea of its own, the generated master TTFs will also not contain any OTL tables.

Just to clarify: this means using fontmake to generate a TTF specifically at the location where the sparse master is, or just any TTF including the VF-TTF will have no opentype features?

Anchor positions is perhaps the bigger issue here. If you want an intermediate glyph outline, you'd want any anchors to also interpolate with it.

In the current project, I have glyphs in the sparse master where I am only moving the anchor, same with component glyphs

[weiweihuanghuang]

@anthrotype I tried your suggestion and the VF generates now! I have recreated the structure of the project with my own Work Sans here to reproduce the same issues: WS.zip

WorkSans-Sparse.designspace recreates the above issue. WorkSans-Sparse-EmptyFeatureWriters.designspace uses the sparse UFO with the empty feature writer.

Edit: is this related to the above discussion? I tried to move a anchor in a glyph in the sparse master with the empty feature writer, and that wasn't reflected in the end vf-ttf

[arrowtype]

Awesome; can’t wait to try this workflow!

[madig]

I tried to move a anchor in a glyph in the sparse master with the empty feature writer, and that wasn't reflected in the end vf-ttf

Probably. Anchors exist only in the GPOS table, which is written by feature writers. Currently, you can only use sparse masters and layers to change outlines and spacing and nothing else.

Not sure what is needed to make this work more broadly...

[twardoch]

One thing I'll add: FontLab 7 lers you do sparse masters but the way we build VF is that we feed a designspace and our own TTFs into varLib. One thing we had to change was to build GSUB only for the main master, otherwise varLib was trying to varmerge incompatible GSUBs and failed.

For GPOS, somehow it works with kern, because of the way we build the kern feature I guess, but other GPOS feature varmerging is still a big issue.

See: https://github.com/googlefonts/fontmake/issues/609

On Tue, 3 Dec 2019 at 11:29, Nikolaus Waxweiler notifications@github.com wrote:

I tried to move a anchor in a glyph in the sparse master with the empty feature writer, and that wasn't reflected in the end vf-ttf

Probably. Anchors exist only in the GPOS table, which is written by feature writers. Currently, you can only use sparse masters and layers to change outlines and spacing and nothing else.

Not sure what is needed to make this work more broadly...

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/googlefonts/fontmake/issues/607?email_source=notifications&email_token=AAD6XRH4PIU3APWWJNZZKQLQWYYJXA5CNFSM4JTFJF22YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEFY4FBI#issuecomment-561103493, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAD6XRF5E75JFHK6WNCYRZDQWYYJXANCNFSM4JTFJF2Q .

[twardoch]

EDIT: I’ve rewritten my comment into https://github.com/googlefonts/fontmake/issues/609

[weiweihuanghuang]

Just noticed using the above project sample I shared to generate an instance, ~I get some 16,000 kerning pairs, whereas generating the same instance without the intermediate masters in the above project, I get around 2900~ [Edit: see below comment]... I wonder if that has to do with this issue https://github.com/googlefonts/fontmake/issues/625

[weiweihuanghuang]

Actually I looked into it more, it seems regardless of whether the sparse intermediate master is part of the designspace or not, the TTF has a grossly overblown number of kerning pairs compared to the UFO.

Strangely, there's still a discrepancy between the results with and without the sparse intermediate master – even though the sparse master has no kerning:

With sparse master: .UFO has 2338 pairs and the TTF has 16101 pairs Without the sparse master: .UFO has 2885 pairs and the TTF has 19102 pairs.

[madig]

How do you measure the number of kern pairs? Do you see a pattern which glyphs get new kern pairs (e.g. glyph2glyph or class2class kerning)? Are these along the interpolation axes?

[kontur]

I think I've run into this as well when compiling a variable font from UFOs extracted from Glyphsapp that have brace layers.

The file is a basic 2 master wght setup but some glyphs have swashes where the extent of the swash is exposed in the VF as swsh axis and defined solely as brace layers for those few glyphs where swashes are applicable. In the designspace there are two weight masters, as well as the two brace layer masters for the swash axis extrema. Those compiled fonts don't interpolate the mark positions on the swsh axis / brace layers (e.g. where a mark anchor may visually move as the swash extends).

My crude solution was to make the brace layers explicit standalone UFOs (copy weight master UFOs, copy brace layer glyphs to regular glyphs, remove layers, reference standalones in designspace).

[anthrotype]

it would help if you could provide a reproducer. I also suggest trying with older versions of glyphsLib in case this is some sort of regression (pip install glyphsLib=={version}, see https://github.com/googlefonts/glyphsLib/releases)

[kontur]

mark-debug-test-case.zip

This includes a glyph source and faulty VF as described above. The VF is compiled in two steps, first use fontmake to extract, then I modified the designspace (the designspace doesn't get extended to the swsh max if those are only encountered in brace layers, but nvm that), then compile VF with fontmake. The files, a test string and requirements.txt are inside the archive with full information.

[anthrotype]

(the designspace doesn't get extended to the swsh max if those are only encountered in brace layers, but nvm that)

maybe "Axis Mappings" custom parameter would fix that

[anthrotype]

btw, thanks for the file I'll take a look soon

[anthrotype]

some glyphs have swashes where the extent of the swash is exposed in the VF as swsh axis and defined solely as brace layers for those few glyphs where swashes are applicable

isn't that the use case for so-called Virtual Masters? I think we now support those

[anthrotype]

see https://github.com/googlefonts/glyphsLib/pull/955

[kontur]

maybe "Axis Mappings" custom parameter would fix that

Ah yea, could be. In this case the designspace file is tweaked in other regards as well with <rules> and modifying the instance list from the default glyphsLib spits out, so not an issue.

isn't that the use case for so-called Virtual Masters? I think we now support those

Need to check that, first I've seen about it 👍