Add checks for glyph sets being complete and up to date

[davelab6]

In Alexei's https://github.com/google/fonts/pull/556 (and discussion here) he rejigged the Cyrillic glyph sets. Therefore we should have some FB checks to ensure that the TTFs we are onboarding are using the latest glyphsets, since we'll want to know about it to bring them all up to date in the future.

[m4rc1e]

Surely this should apply to all our charsets and not just Cyrillic?

[davelab6]

This should be done soon so moved to 0.2.0

If this is to be be done with pyfontaine, that means updating pyfontaine to use the stuff @graphicore is doing on the glyph set definitions in the /google/fonts repo...

[felipesanches]

In this issue description, where you say "(and discussion here) " the link seems broken:

https://groups.google.com/forum/#!msgid/googlefonts-discuss/CADUrt2EuXciPtRvxV4HojXCYeym7gpthHBKVdLoueVyaL6-wng%2540mail.gmail.com

@davelab6 can you please check that URL and possibly provide a good one so that I can read the referenced discussion?

[felipesanches]

The closest I could find by searching Alexei's postings to the google-fonts-discuss mailing list was his "Font Quality Improvement Project — Alexei's Worklog" but that's hundreds of messages...

@davelab6 could you perhaps pinpoint exactly where is the relevant discussion in the mailing list?

[felipesanches]

(or maybe @alexeiva himself could help here as well :-D)

[alexeiva]

The latest discussion on glyphsets is here:

https://github.com/google/fonts/pull/685#discussion_r105740288

Lasse has updated the documentation and unified filter list vs nam files consistency.

On Mar 23, 2017 02:41, "Felipe Corrêa da Silva Sanches" < notifications@github.com> wrote:

(or maybe @alexeiva https://github.com/alexeiva himself could help here as well :-D)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/googlefonts/fontbakery/issues/1189#issuecomment-288573353, or mute the thread https://github.com/notifications/unsubscribe-auth/AP3Q6X21Dk8iwSZQ2Pf81mUpkf7-JoCdks5robGogaJpZM4LScxB .

[graphicore]

@felipesanches to read the *.nam files you should use def codepointsInNamelist which returns a set of all unicode codepoints (as integers) in the Namelist and its includes (dont use unique_glyphs = False

To get all unicodes from a font you can have a look at def namelist_from_font Where the most subtle information is that not all cmap are Unicode encoded, hence: if not cmap.isUnicode(): continue.

For your kind of tests it may be helpful to know which Glyphsets should be supported by a font, I don't know where you would get that information from.

[felipesanches]

@davelab6, @behdad, @graphicore, @m4rc1e and @alexeiva:

I have ported the Google Fonts Glyphsets definitions from https://github.com/google/fonts/tree/master/tools/encodings/GF%20Glyph%20Sets into pyfontaine so that it can be used to check glyph coverage status on fontbakery checks.

This is the pull request with the port: https://github.com/davelab6/pyfontaine/pull/93

While doing so, I removed the old gwf_* glyphsets since I presume they are now deprecated. If that's not the case, I can bring them back.

These are the glyphsets that I ported into pyfontaine:

google_cyrillic_historical            Google Fonts: Cyrillic Historical
google_cyrillic_plus                  Google Fonts: Cyrillic Plus
google_cyrillic_plus_locl             Google Fonts: Cyrillic Plus (Localized Forms)
google_cyrillic_pro                   Google Fonts: Cyrillic Pro
google_greek_ancient_musical_symbols  Google Fonts: Greek Ancient Musical Symbols
google_greek_archaic                  Google Fonts: Greek Archaic
google_greek_coptic                   Google Fonts: Greek Coptic
google_greek_core                     Google Fonts: Greek Core
google_greek_expert                   Google Fonts: Greek Expert
google_greek_plus                     Google Fonts: Greek Plus
google_greek_pro                      Google Fonts: Greek Pro
google_latin_core                     Google Fonts: Latin Core
google_latin_expert                   Google Fonts: Latin Expert
google_latin_plus                     Google Fonts: Latin Plus
google_latin_plus_optional            Google Fonts: Latin Plus (Optional Glyphs)
google_latin_pro                      Google Fonts: Latin Pro
google_latin_pro_optional             Google Fonts: Latin Pro (Optional Glyphs)

And these are the remaining gwf_ glyphsets that I did not remove because they do not have glyph-list (.nam) files on google/fonts git repo yet:

gwf_arabic         Google arabic
gwf_extras         Google extras
gwf_vietnamese     Google vietnamese

Once we merge that into pyfontaine, I'll be writing specific checks in fontbakery to error if a given family fails to provide all glyphs from a PLUS glyphset and checks that spit out only WARNINGS if a family fails to implement glyphs from EXPERT/PRO sets.

I think that's the expected behaviour based on the following statement available on https://github.com/google/fonts/blob/master/tools/encodings/GF%20Glyph%20Sets/README.md :

"Three levels of glyph sets were developed in June 2016 as a new baseline standard for fonts in the Google Fonts library. Exising fonts in the library can be upgraded to these as part of a drive towards new quality standards. All new fonts submitted to the library must now support the Plus level as a minumum requirement."

Attached is an example of pyfontaine being used to check the Google Fonts Greek Pro glyph coverage of Rubik Bold (it scores at 87%).

cheers, Felipe

[felipesanches]

Work on this issue is currently blocked by a pending pull request on the upstream pyfontaine repo: davelab6/pyfontaine#93 Who can merge that? It that only you, @davelab6 ?

[davelab6]

Merged