URGENT: Critical error with GDEF table

[davelab6]

To reproduce:

# install homebrew
ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)";
sudo xcodebuild -license;
# install ots
brew tap davelab6/webfonttools; 
brew update; 
brew install ots --HEAD;
# lint the font
ot-sanitise build/Hubballi-Regular.ttf 

Currently this fails with:

ERROR at src/layout.cc:204 (ParseLookupTable)
ERROR: Layout: Lookup flags require GDEF table, but none was found: 8
ERROR at src/layout.cc:1346 (ParseLookupListTable)
ERROR: Layout: Failed to parse lookup 7
ERROR at src/gsub.cc:618 (ots_gsub_parse)
ERROR: GSUB: Failed to parse lookup list table
ERROR at src/ots.cc:669 (ProcessGeneric)
Failed to sanitise file!

This is fatal and means the font won't work in Chrome and Firefox.

[davelab6]

The OTF doesn't have a problem, though

[davelab6]

@erinmclaughlin ping

[davelab6]

@erinmclaughlin :)

[davelab6]

@erinmclaughlin any luck with this? :)

[erinmclaughlin]

Hey! I contacted Nicolas, who said he'd only be available this weekend to take a look. I'm sorry I've been so busy! Now... trying to get the Bengali thing done :)On March 18, 2016 at 1:19 AM Dave Crossland notifications@github.com wrote:@erinmclaughlin any luck with this? :)—You are receiving this because you were mentioned.Reply to this email directly or view it on GitHub  

[davelab6]

Okay cool :) We can't launch Hubballi without it, but anytime early next week is good :)

[zar-nicolas20]

I am on it now, just read what the issue is. I will take a look to it

[zar-nicolas20]

I think that the issue is that there is not base glyph class defined in the GDEF

[davelab6]

Fixed in https://github.com/erinmclaughlin/Hubballi/commit/6e248b25991410f770c7701e96cd3cfd33239c55

[davelab6]

@erinmclaughlin I'm sad to say that the error persists in the latest TTF :(

$ git pull https://github.com/erinmclaughlin/Hubballi.git
From https://github.com/erinmclaughlin/Hubballi
 * branch            HEAD       -> FETCH_HEAD
Already up-to-date.
$ ot-sanitise build/Hubballi-Regular.ttf 
ERROR at src/layout.cc:204 (ParseLookupTable)
ERROR: Layout: Lookup flags require GDEF table, but none was found: 8
ERROR at src/layout.cc:1346 (ParseLookupListTable)
ERROR: Layout: Failed to parse lookup 7
ERROR at src/gsub.cc:618 (ots_gsub_parse)
ERROR: GSUB: Failed to parse lookup list table
ERROR at src/ots.cc:669 (ProcessGeneric)
Failed to sanitise file!

[zar-nicolas20]

I will take a look to see what is defined in the feature file, what is in fact compiled in the font, and what makeotf is being used.

Sent from my iPhone

On Mar 25, 2016, at 6:29 AM, Dave Crossland notifications@github.com wrote:

@erinmclaughlin I'm sad to say that the error persists in the latest TTF :(

$ git pull https://github.com/erinmclaughlin/Hubballi.git From https://github.com/erinmclaughlin/Hubballi

• branch HEAD -> FETCH_HEAD Already up-to-date. $ ot-sanitise build/Hubballi-Regular.ttf ERROR at src/layout.cc:204 (ParseLookupTable) ERROR: Layout: Lookup flags require GDEF table, but none was found: 8 ERROR at src/layout.cc:1346 (ParseLookupListTable) ERROR: Layout: Failed to parse lookup 7 ERROR at src/gsub.cc:618 (ots_gsub_parse) ERROR: GSUB: Failed to parse lookup list table ERROR at src/ots.cc:669 (ProcessGeneric) Failed to sanitise file! — You are receiving this because you commented. Reply to this email directly or view it on GitHub

[zar-nicolas20]

There is no changes in the GDEF table yet. I see a @BASE and a @LIGA class, but they are not declared in the GDEF.

The GDEF should looks like this:

table GDEF {
  GlyphClassDef @BASE, @LIGA, @COMBINING_MARKS, ;
} GDEF;

That should works

[erinmclaughlin]

oh my GOSH I must be going insane... I must not have hit save after I changed the GDEF the last time. I am so so so sorry for wasting your time, both of you!! Ok I just uploaded a revised version with new fonts generated

[zar-nicolas20]

You are more than welcome

Best,

Nicolás

On Mar 26, 2016, at 10:17 PM, Erin McLaughlin notifications@github.com wrote:

oh my GOSH I must be going insane... I must not have hit save after I changed the GDEF the last time. I am so so so sorry for wasting your time, both of you!! Ok I just uploaded a revised version with new fonts generated

— You are receiving this because you commented. Reply to this email directly or view it on GitHub

[davelab6]

It is not working:

$ ot-sanitise github.com/davelab6/Hubballi/build/Hubballi-Regular.ttf 
ERROR at src/layout.cc:204 (ParseLookupTable)
ERROR: Layout: Lookup flags require GDEF table, but none was found: 8
ERROR at src/layout.cc:1346 (ParseLookupListTable)
ERROR: Layout: Failed to parse lookup 7
ERROR at src/gsub.cc:618 (ots_gsub_parse)
ERROR: GSUB: Failed to parse lookup list table
ERROR at src/ots.cc:669 (ProcessGeneric)
Failed to sanitise file!

[zar-nicolas20]

I saw in the members of @BASE some marks and non letters characters such as .notdef, zerowithnotjoiner, acutecomb etc. I will try to purge @BASE leaving only real base letters, removing control characters, numbers, punctuation, space characters, mathematical operators and marks. Best,

[davelab6]

Thanks boss!! :D

[erinmclaughlin]

Very good suggestion, thank you so much Nicolas :) I just cleaned it up. Want to give this a try, Dave?

[davelab6]

I'm sorry to say it is still not working.

$ ot-sanitise build/Hubballi-Regular.ttf 
ERROR at src/layout.cc:204 (ParseLookupTable)
ERROR: Layout: Lookup flags require GDEF table, but none was found: 8
ERROR at src/layout.cc:1346 (ParseLookupListTable)
ERROR: Layout: Failed to parse lookup 7
ERROR at src/gsub.cc:618 (ots_gsub_parse)
ERROR: GSUB: Failed to parse lookup list table
ERROR at src/ots.cc:669 (ProcessGeneric)
Failed to sanitise file!

Are you able to reproduce this yourself? Here is how to install it:

# install homebrew
ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)";
sudo xcodebuild -license;
# install ots
brew tap davelab6/webfonttools; 
brew update; 
brew install ots --HEAD;
# lint the font
ot-sanitise build/Hubballi-Regular.ttf 

[erinmclaughlin]

Hey Dave! I just rebuilt the font in RoboFont with a fresh file, and the .ttf wouldn't generate. Which is actually perhaps a good thing, as it can maybe help us figure out what's going on. Here's the error. Can you translate this into English for me? :) hehee

Traceback (most recent call last): File "lib/fontObjects/doodleFontCompiler/ttfCompiler.pyc", line 486, in compile File "ufo2fdk/makeotfParts.pyc", line 64, in compile File "ufo2fdk/makeotfParts.pyc", line 79, in setupFile_outlineSource File "lib/fontObjects/doodleFontCompiler/ttfCompiler.pyc", line 248, in compile File "/Applications/RoboFont.app/Contents/Resources/lib/python2.7/fontTools/ttLib/__init__.py", line 200, in save File "/Applications/RoboFont.app/Contents/Resources/lib/python2.7/fontTools/ttLib/__init__.py", line 596, in _writeTable File "/Applications/RoboFont.app/Contents/Resources/lib/python2.7/fontTools/ttLib/__init__.py", line 608, in getTableData File "/Applications/RoboFont.app/Contents/Resources/lib/python2.7/fontTools/ttLib/tables/_p_o_s_t.py", line 47, in compile File "/Applications/RoboFont.app/Contents/Resources/lib/python2.7/fontTools/ttLib/tables/_p_o_s_t.py", line 149, in encode_format_2_0 UnicodeDecodeError: 'ascii' codec can't decode byte 0xe9 in position 5: ordinal not in range(128)

[davelab6]

On 30 March 2016 at 15:12, Erin McLaughlin notifications@github.com wrote:

UnicodeDecodeError: 'ascii' codec can't decode byte 0xe9 in position 5: ordinal not in range(128)

You have a string somewhere that is not ASCII, perhaps in the POST table.

[erinmclaughlin]

Ok! Must have had an emdash instead of a hyphen or something, no clue, but it's generating now. Can you try the new test fonts in the Generated Test Fonts folder?

[davelab6]

It is still broken :(

$ ot-sanitise Hubballi/UFO/GENERATED\ TEST\ FONTS/Hubballi-test.ttf 
ERROR at src/layout.cc:204 (ParseLookupTable)
ERROR: Layout: Lookup flags require GDEF table, but none was found: 8
ERROR at src/layout.cc:1346 (ParseLookupListTable)
ERROR: Layout: Failed to parse lookup 7
ERROR at src/gsub.cc:618 (ots_gsub_parse)
ERROR: GSUB: Failed to parse lookup list table
ERROR at src/ots.cc:669 (ProcessGeneric)
Failed to sanitise file!

[erinmclaughlin]

I've also noticed that at some point along the way, my "lookupflag IgnoreMarks;" virama fix stopped working, too. :( Aghghhh. I'll keep backtracking and try to figure it out :( I totally don't understand why this font doesn't work, but my others do...

[erinmclaughlin]

Hey Dave! Ok, so I tried recreating Hubballi in Glyphs, and exported a .ttf from that. I also brought the Glyphs-built .UFO back into Robofont and exported that again. Could you open the generated fonts folder and test all three of the .ttfs? Do any of those work in the validator?

The only problem now is that I still am not getting the virama issue to work. It might be because I'm unsure of how to set up the GDEF in Glyphs. I created a custom GlyphData.xml file to assign the non-unicode Kannada glyphs to either "Letter" or "Mark" categories, but that didn't seem to do the trick. Can you point me where to look for this?

[davelab6]

These 2 are broken:

$ ot-sanitise UFO/GENERATED\ TEST\ FONTS/Hubballi-Regular-Glyphs-exp.ttf 
ERROR at src/layout.cc:204 (ParseLookupTable)
ERROR: Layout: Lookup flags require GDEF table, but none was found: 8
ERROR at src/layout.cc:1346 (ParseLookupListTable)
ERROR: Layout: Failed to parse lookup 10
ERROR at src/gsub.cc:618 (ots_gsub_parse)
ERROR: GSUB: Failed to parse lookup list table
ERROR at src/ots.cc:669 (ProcessGeneric)
Failed to sanitise file!

$ ot-sanitise UFO/GENERATED\ TEST\ FONTS/Hubballi-test.ttf 
ERROR at src/layout.cc:204 (ParseLookupTable)
ERROR: Layout: Lookup flags require GDEF table, but none was found: 8
ERROR at src/layout.cc:1346 (ParseLookupListTable)
ERROR: Layout: Failed to parse lookup 7
ERROR at src/gsub.cc:618 (ots_gsub_parse)
ERROR: GSUB: Failed to parse lookup list table
ERROR at src/ots.cc:669 (ProcessGeneric)
Failed to sanitise file!

But this one basically works; the max_zones issue can be hotfixed with ttx.

$ ot-sanitise UFO/GENERATED\ TEST\ FONTS/Hubballi-Regular-Glyphs.ttf 
ERROR at src/maxp.cc:57 (ots_maxp_parse)
WARNING: maxp: bad max_zones: 0

This is not a problem, and ot can 'hotfix' it like this:

ot-sanitise Hubballi-Regular-Glyphs.ttf Hubballi-Regular-Glyphs-fixed.ttf

[davelab6]

Can you point me where to look for this?

I'm not so familiar. I think James Puckett or Nicolas would know.

[erinmclaughlin]

Yay! A tiny shred of progress! I'll ask them :)

[erinmclaughlin]

update: Georg is helping me with this... inching closer :)

[zar-nicolas20]

Maybe if you use the flag IgnoreMark in a feature, try to defined a markClass, otherwise the OT compiler won’t have the information to write the MarkAttachClassDef table, that is required when the OT Layout shaping hit a lookup with “ignore Marke” flag.

[zar-nicolas20]

The current OTF seems working:

=============================================
    Hubballi-Regular-Robofontmu.ttf
ERROR at src/layout.cc:204 (ParseLookupTable)
ERROR: Layout: Lookup flags require GDEF table, but none was found: 8
ERROR at src/layout.cc:1346 (ParseLookupListTable)
ERROR: Layout: Failed to parse lookup 7
ERROR at src/gsub.cc:618 (ots_gsub_parse)
ERROR: GSUB: Failed to parse lookup list table
ERROR at src/ots.cc:669 (ProcessGeneric)
Failed to sanitise file!
=============================================
    Hubballi-Regular-Robofontmu.otf
=============================================

[zar-nicolas20]

@erinmclaughlin Open Terminal, and go to the directory in which the fonts are, something like:

cd ~/Huballi/sources/generated\ test\ fonts/

type ls in order to verify that you can see Hunnalli-Regular_Robofontmu.otf and its TTF counterpart.

Then type the following command in order to inject the GDEF of the OTF into the TTF:

ttx -t 'GDEF' -o Hubballi.GDEF.ttx Hubballi-Regular-Robofontmu.otf ; ttx -m Hubballi-Regular-Robofontmu.ttf Hubballi.GDEF.ttx ; rm Hubballi.GDEF.ttx ; mv Hubballi.GDEF.otf Hubballi-Regular-Robofontmu.ttf

That will work. Attached a TTF that pass OTS Hubballi-Regular-Robofontmu.ttf.zip

[erinmclaughlin]

@zar-nicolas20 Hi Nicolas! Thanks so much for keeping an eye on this and continuing to help!! I appreciate it SO much. Unfortunately, I just tested the new .ttf that you posted in Firefox and the Virama feature is broken again. Did that command attempt to copy the .otf's GDEF into the .ttf? (Sorry that I don't know what these commands do.) I wonder what we should try next?

[erinmclaughlin]

Maybe I just need to find some examples of recently-generated fonts that have unusual marks and "ignoremarks" lookups that actually pass the test, so I can see what I'm doing wrong... Do you think this has anything to do with the fact that I am not actually using anchors for any of the marks? (Since I built precomposed consonant+vowel combination glyphs, and the marks are just used for input for the substitutions?)

[zar-nicolas20]

No anchors, hmm that changes everything.

The script that I passed you dumps the GDEF from the OTF and merge it to the TTF. The strange thing is that the TTF is lacking of a GDEF when the OTF have one.

Try to generate a OTF in which the Virama works, then generate a TTF and use the script that I send you, one time writing GDEF and a second time writing GSUB.

Best,

Nicolás

On Apr 4, 2016, at 8:23 PM, Erin McLaughlin notifications@github.com wrote:

Maybe I just need to find some examples of recently-generated fonts that have unusual marks and "ignoremarks" lookups that actually pass the test, so I can see what I'm doing wrong... Do you think this has anything to do with the fact that I am not actually using anchors for any of the marks? (Since I built precomposed consonant+vowel combination glyphs, and the marks are just used for input for the substitutions?)

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub

[zar-nicolas20]

I type wrong the command, tomorrow I will fix it

[zar-nicolas20]

First check if your OTF is working as expected, if so, in addition to inject the GDEF from the OTF into the TTF, you can also inject the GDEF and GSUB, the way to do that will be the following; Open terminal, go to the same directory in which the OTF and the TTF are placed, then type the following command:

ttx -t 'GDEF' -t 'GSUB' -t 'GPOS' -o Hubballi.OTtables.ttx Hubballi-Regular-Robofontmu.otf ; 
ttx -m Hubballi-Regular-Robofontmu.ttf Hubballi.OTtables.ttx ; 
rm Hubballi.OTtables.ttx ; 
mv Hubballi.OTtables.otf Hubballi-Regular-Robofontmu.ttf

And press enter.

[erinmclaughlin]

Darn! Still didn't work. So maybe this does have to do with the construction of the glyphs, that I didn't use anchors with these marks? Luckily, Georg was able to recreate my file in Glyphs and did some alterations to the GlyphData.xml so that it's now working. I still wish that we could figure out why it's not working in Robofont! Maybe someday we can revisit this when we have more time! Thank you SO SO much for helping me.

[davelab6]

What's the latest status of this? :)

[zar-nicolas20]

I saw that the TTF lacks GDEF at all. That is the problem with font parser. Besides that I think Erin is working with something with Virama and an ignore mark flag in a lookup.

I post a work around merging the GDEF of the OTF into the TTF using TTX.

Best,

Nicolás

On Apr 14, 2016, at 12:45 PM, Dave Crossland notifications@github.com wrote:

What's the latest status of this? :)

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub

[erinmclaughlin]

Hi guys! The font export is working from glyphs - I was cleaning up file, re-composing accented glyphs. So sorry for delay, will update this as soon as I can, hopefully tonight On April 14, 2016 at 11:34 PM José Nicolás Silva Schwarzenberg notifications@github.com wrote:I saw that the TTF lacks GDEF at all. That is the problem with font parser. Besides that I think Erin is working with something with Virama and an ignore mark flag in a lookup. I post a work around merging the GDEF of the OTF into the TTF using TTX. Best, Nicolás > On Apr 14, 2016, at 12:45 PM, Dave Crossland notifications@github.com wrote: > > What's the latest status of this? :) > > — > You are receiving this because you were mentioned. > Reply to this email directly or view it on GitHub > —You are receiving this because you were mentioned.Reply to this email directly or view it on GitHub  

[erinmclaughlin]

Hi Dave, sorry again for the lateness. One setback (among other life things) is that when Georg re-created my file in Glyphs, sidebearings got all messed up and kerning didn't come along with it. Having to find/replace all of the glyphnames to match what is used in Glyphs. Taking a long time :) I will try to be finished with it tonight.

[davelab6]

thanks erin!! :)