GDEF/GPOS output different depending on source format

[madig]

1. git clone -b convert-to-ufo --depth 1 https://github.com/madig/cantarell-fonts
2. Generate fonts with Glyphs from src/Cantarell.glyphs, ttx -t GDEF Cantarell-Regular.otf
3. Generate fonts with fontmake -g Cantarell.glyphs -i -o otf --output-dir /tmp/fm-g, ttx the GDEF table
4. Generate fonts with glyphs2ufo Cantarell.glyphs && fontmake -m Cantarell.designspace -i -o otf --output-dir /tmp/fm-d, ttx the GDEF table

fontmake/ufo2fts' generated table differs depending on the source format. I'm attaching all three ttx'ed GDEF table dumps.

The same diff occurs in the GPOS table (<BaseCoverage Format="2">, etc.).

gdef-tables.zip

[madig]

Also, in the name table, going from the .glyphs file I get nameID1="Cantarell", going from .designspace I get nameID1="Cantarell Regular" plus nameID16="Cantarell". Not sure if that may be something in glyphsLib, though.

[anthrotype]

the source of the difference seems to be the fact that when creating the UFOs with glyphs2ufo script, by default glyphsLib does not generate a GDEF table override in the UFO's features.fea based on the GlyphData.xml. So the GDEF that you see in the generated OTFs is what feaLib synthetizes from looking at the way glyphs are used in the mark feature's lookups. https://github.com/fonttools/fonttools/blob/32aef4fc880f4c8e9c564ea1f48849fe1f06cbe7/Lib/fontTools/feaLib/builder.py#L487-L503

When generating from fontmake directly from a glyphs source file, the GDEF table override is generated in the intermediate UFOs and thus the OTFs will show such differences in GDEF and correspondingly in the GPOS.

Currently, and counter-intuitively, you need to pass to glyphs2ufo the option --no-preserve-glyphsapp-metadata to make it generate that GDEF table in features.fea based on the logic from GlyphData.xml database. Honestly I wasn't aware of this difference between fontmake and glyphs2ufo until now.

Probably we need to add an explicit option that controls this GDEF table generation in glyphsLib and expose it in the glyphs2ufo command line.

[madig]

glyphsLib decides on GDEF inclusion here: https://github.com/googlei18n/glyphsLib/blob/master/Lib/glyphsLib/builder/features.py#L92

Jany says:

Yes, I did that because I don't parse/preserve any existing GDEF from the UFOs so after roundtrip it would be changed, and because if the source UFO didn't have a GDEF to begin with, then it would get one from the roundtrip (which I thought was not expected)

Edit, addition:

Actually, maybe an existing GDEF would be preserved: https://github.com/googlei18n/glyphsLib/blob/master/Lib/glyphsLib/builder/features.py#L524-L527 There's no test with a GDEF in https://github.com/googlei18n/glyphsLib/blob/master/tests/builder/features_test.py maybe, if you take a decision about this, you could try adding a test with a GDEF in features_test.py

[anthrotype]

What Jany says makes sense. Still, the fact that it is controlled by an option called --no-preserve-glyphsapp-metadata kind of hides it from the user. Maybe it's enough that we add a few lines in the glyphs2ufo help message for that option?

[madig]

We had the idea that maybe we can make writing the table independent of a switch and instead only write the table if we think we need to.

@moyogo @belluzj in what situations does one need to write their own table?

[madig]

I did a few tests with Glyphs:

1. Glyphs only generates a GDEF table when you want to compile a binary
2. If a GDEF table is present in the UFO, it is put into the "Prefix" feature group thing and left untouched (both when exporting to UFO and when making a binary).

Incidentally, glyphsLib behavior matches this unless I'm overlooking something. So it is expected that UFOs don't have GDEF tables, but fontmaking from a Glyphs file should use the information in the file to generate something.

So... leave as is and maybe make an extra switch to explicitly write the GDEF (off by default)?

[madig]

The GDEF of e.g. Cantarell exported by Glyphs is also much larger than the one generated by glyphsLib.

[anthrotype]

from @madig comment in https://github.com/googlei18n/glyphsLib/issues/427#issuecomment-421295996, it looks like one also needs to pass --propagate-anchors option (in addition to --no-preserve-glyphsapp-metadata) to the glyphs2ufo script in order to get the same GDEF table that one would get by doing fontmake -g MyFont.glyphs -o ufo.

[madig]

glyphsLib should probably insert

<key>com.github.googlei18n.ufo2ft.filters</key>
<array>
  <dict>
    <key>name</key>
    <string>propagateAnchors</key>
    <key>pre</key>
    <true />
  </dict>
</array>

into a UFOs lib.plist on round-tripping. Or at least tell the user to consider it.