Subset icon font - Githubissues

caperaven commented 3 years ago

The icon font has all the icons available. I am looking for a way to customise or create a custom font that only contains the icons I use. What tooling do you use to create the current icon fonts?

I am kinda new to the font generation thing but would like to get the same behaviour as using the standard font. That being that the icon name is the identifier of the icon not some ascii code.

The intent of the custom font is for personal use but are there any legal issues I should take note of?

jfbrennan commented 3 years ago

Need the same because the docs advertise 900+ fonts as weighing "in at only 42KB in its smallest woff2 format" but I'm seeing 109KB and no compression on the response in dev tools.

My code:

src: url('https://fonts.gstatic.com/s/materialicons/v97/flUhRq6tzZclQEJ-Vdg-IuiaDsNcIhQ8tQ.woff2') format('woff2');

jfbrennan commented 3 years ago

@caperaven you might want to change the title to better reflect the ask. Perhaps "Customize the icon font set to reduce size"

jfbrennan commented 3 years ago

Dup of https://github.com/google/material-design-icons/issues/564, which is 5 years old 😢

caperaven commented 3 years ago

I am happy to build the font myself, I am just unsure of what the best tooling is to use for this. Any suggestions?

rsheeter commented 3 years ago

(took the liberty of amending the title)

Figure out the glyph ids for the icons you want to keep (because using text= may retain other icons reachable with the same text, e.g. alarm_on might keep alarm as well) and then subset by glyph id. For example, you could use https://rsheeter.github.io/font101/#hb-shape to get the glyph id and then https://rsheeter.github.io/font101/#pyftsubset to subset the font. This could all be packaged up into a nice cli that takes a list of icon names if you need to do it repeatedly.

Edit: if you plan to deliver to the web you will want to compress the resulting font; https://rsheeter.github.io/font101/#web-serving has some suggestions.

tphinney commented 2 years ago

Looks like this was resolved long ago!

EskelCz commented 1 year ago

@tphinney Actually not really, because it doesn't work with the new Material Symbols. I tried all the options of pyftsubset for an hour, but the ligatures are getting lost in the process. :( Fontsquirrel and fontello doesn't work either. The font has over 3MB, therefore the subsetting is truly necessary.

tphinney commented 1 year ago

Probably the ligatures are getting lost because you aren’t including the glyphs that are needed to input the ligatures. Besides the desired symbols you would need a-z, space, underscore, and to include the 'rlig' feature.

EskelCz commented 1 year ago

@tphinney Ah, I see, that makes sense, thanks.

Tried for another hour and whenever I add U+61-7A (supposedly a-z), the ligatures are there but the font balloons to 2.5MB (from 30kB for just the 30 or so icons I need).

At this point I decided to go with link to fonts.googleapis.com, which download about 40kB for the same font. No idea how, but it works. Still I would prefer self-hosting, if I could manage to make the subset.

tphinney commented 1 year ago

Note: realized I was forgetting the zero through nine characters, also needed to get at the names of some glyphs.

So, it seems like you are now getting EVERY ligature that can be formed with those glyphs. Which would be nearly all of them. (Except those needing characters you don't need.

Whatever tool you use to subset needs to both: 1) support maintaining the ligation code 2) allow you to decide which ligatures you need instead of grabbing all possible ones

I don't know if pyftsubset can do that for you. If not, you need something that can. Unfortunately all this is a bit of a “corner case” as far as font subsetting goes. It is an unusual situation where one wants some but not all of the ligatures that can derive from the available characters.

EskelCz commented 1 year ago

@tphinney Thanks for the explanation. I haven't found any other tools that can do that programmatically. I think this calls for a feature of the website. The back-end must have such code already, since it's somehow working when used from the api - it's not serving the entire 3MB font file. Are feature requests tracked anywhere?

tphinney commented 1 year ago

Yes. This is the place for external feature requests. There is also an internal Google database for such things. I can create an issue there next week. I am traveling at the moment.

EskelCz commented 1 year ago

@tphinney Sorry to bother again but I just noticed that the API now returns the large file as well. I'd love to use the variable icon font but the size doesn't seem worth it.

This is my integration code:

<link href="https://fonts.googleapis.com/css2?family=Material+Symbols+Rounded:opsz,wght,FILL,GRAD@48,100..700,1,0..200" rel="stylesheet" />

Can you please escalate this somehow? Thanks

EskelCz commented 1 year ago

@tphinney Can you please reopen this issue, or should I open a new one about the variable symbol subsetting? I feel like it's a huge waste for such an awesome asset to be basically unusable in production.

jfbrennan commented 1 year ago

Yes, please reopen. Came back here after 2 years sure I would find Google has indeed solved this by now. Darn.

No app uses 3,000 icons. The smallest configuration I've been able to get yields a 323kb woff2 file (using family=Material+Symbols+Outlined:FILL@0..1). 323kb is relatively small for 3,000 freaking icons plus filled versions, but that's still far too big for devs who don't like building 10MB React SPAs🤦‍♂️

Ideal solution: CLI where developers can feed a list of icon names, e.g. home done add expand_more, a style outline, and a list of options, e.g. --fill:0..1, and out pops a new Material Symbols .woff2 variable font with only those icons. I could go from 323kb to like 12kb.

If anyone has ever achieved exactly this, PLEASE reveal your secrets! Like maybe the folks at Google working on Material Symbols who know the tooling/config used to build the full Material Symbols font files. Can ya throw us a bone here?!

rsheeter commented 1 year ago

Reopening because it would be a good feature to have.

If you want to DIY I think what you want to do is roughly:

Identify what glyphs - not codepoints - you want to keep
- likely using hb-shape, https://rsheeter.github.io/font101/#hb-shape
Subset specifying glyph-ids taken from ^
- instead of specifying unicodes or text as shown at https://rsheeter.github.io/font101/#pyftsubset, specify --gids
- https://github.com/fonttools/fonttools/blob/02784f3806708454a59c77b8e19dae0b4917f4e5/Lib/fontTools/subset/__init__.py#L63

I'd have to try it to be sure the ligatures would be retained, it's possible you'd end up with just PUA codepoint access. For context, a Google-style icon font typically lets you get the icon glyph by either a PUA or a ligature. If you end up with just PUAs you'd have to rebuild the ligatures.

This could certainly be packaged up into a little tool or exposed by an API endpoint but I believe nobody has sat down and done it so far.

jfbrennan commented 1 year ago

Thanks @rsheeter, will try to look into all that. It's a totally new space to me.

PUA = "Private Use Area" yeah? ligature = the icon "strings", e.g. home? Those characters are kind of like internally mapped to a glyph, yeah?

Tool creators do seem to be struggling to provide a solution, but it is nevertheless a need in the web community https://github.com/dzhuang/subset-iconfont/issues/81

rsheeter commented 1 year ago

PUA = "Private Use Area" yeah?

Yes. On https://fonts.google.com/icons?selected=Material+Symbols+Outlined:menu:FILL@0;wght@400;GRAD@0;opsz@24 this is displayed as codepoint. That codepoint maps directly to the glyph via cmap as described in https://rsheeter.github.io/font101/#glyph-ids-and-the-cmap-table.

ligature = the icon "strings", e.g. home? Those characters are kind of like internally mapped to a glyph, yeah?

One extra indirection, a ligature maps a series of glyphs to another glyph so "menu" would resolve to glyphs and then those glyphs would be replaced by the glyph for the menu icon.

https://learn.microsoft.com/en-us/typography/opentype/spec/gsub#lookuptype-4-ligature-substitution-subtable

Tool creators do seem to be struggling to provide a solution

All the parts are available but I appreciate that doesn't mean it's trivial to assemble them.

rsheeter commented 1 year ago

Apparently I said I would reopen but didn't actually do it

tphinney commented 1 year ago

I agree that “good” subsetting of Material Symbols would be lovely and even important functionality for Google to provide. It is… not trivial, and especially not if adding it to Google’s font-serving capabilities.

Another thing that any subsetter needs to do to maintain full functionality is, IF the subset includes the 'FILL' axis, maintain alt glyphs that get swapped at >99% fill. This is needed to ensure that fully-filled glyphs render correctly, especially (1) when multiple axes are at non-default positions; and/or (2) Apple’s rasterizer is being used, as is commonly the case on iPhone and macOS. (Note to Googlers: there are additional wrinkles on this, if we are dealing with the internal Google Symbols version of the font.)

Note that the way these alts are implemented is not supported in all environments (notably not working in Figma and Adobe desktop apps), though they work in all browsers AFAIK.

rsheeter commented 1 year ago

Another thing that any subsetter needs to do to maintain full functionality is, IF the subset includes the 'FILL' axis, maintain alt glyphs that get swapped at >99% fill

If we lose the alt glyphs when subsetting that's a bug in the subsetter that we would fix so I think for planning purposes one could assume that works.

rsheeter commented 1 year ago

https://github.com/rsheeter/subset-gf-icons/blob/main/src/subset_gf_icons/subset_gf_icons.py shows finding the glyph ids which one would then feed to subset, etc. I only had a few minutes to play so that's as far as I got :)

jfbrennan commented 1 year ago

Google’s font-serving capabilities

A nicety for sure, but I think a CLI is where the biggest impact is. Frontend teams would want to be able to update their project's list of used icon names in a PR and then have CI (or local tooling) spit out the new icon font file, which would get hosted along with all their other assets.

Pardon my frankness, but don't you guys already have these tools available to you?! Somehow you guys are moving the source SVG files in https://github.com/google/material-design-icons/tree/master/symbols/web through some processor into the final .woff2 file(s) we fetch from fonts.gstatic.com? Can't you just like open-source that tool? I'm sure it's not that simple, but you get the point - something inside Google turns Material Symbols SVGs into a .woff2 file. HAND IT OVER! :)

jfbrennan commented 1 year ago

^ it was an intern who did it all by hand wasn't it?

rsheeter commented 1 year ago

If only the intern would come back!

https://github.com/googlefonts/nanoemoji can turn svgs into fonts with ligatures but not a variable font. I don't have a secret utility that builds the variable font sitting around waiting to be open sourced, it's a bit of a process.

Assembling a CLI that subsets in a manner tuned for Google-style icon fonts seems like the simplest option because all the tools needed are already open. If you add a call to subset to https://github.com/rsheeter/subset-gf-icons/blob/main/src/subset_gf_icons/subset_gf_icons.py then all that's left is a bit of fiddling with ligatures.

rsheeter commented 1 year ago

I had a few minutes to spare and added a subset step to https://github.com/rsheeter/subset-gf-icons. This should now keep the glyphs and ligatures for the specified icons only.

EskelCz commented 1 year ago

I had a few minutes to spare and added a subset step to https://github.com/rsheeter/subset-gf-icons. This should now keep the glyphs and ligatures for the specified icons only.

Thanks for the effort, but I fear there might have been a misunderstanding. In the readme you target /material-design-icons/font/MaterialIcons-Regular.ttf, but I assume we're recently asking for subsetting of the VARIABLE font (Material Symbols). That is /material-design-icons/variablefont/.... There are working subsetting utilities for the classic icons already, but none are working for ligatures of the new ones.

Or am I mistaken and you mean it can it be used for both?

rsheeter commented 1 year ago

you mean it can it be used for both?

Poorly chosen examples ftw. It should work for any Google-style icon font. If not we have a bug to fix. I have only tested with MaterialIcons-Regular.ttf because it's faster to test with and I was in a hurry.

EskelCz commented 1 year ago

@rsheeter Wow, I didn't expect that. That's great, thanks! I managed to generate a 40kb ttf subset (20 icons) which is great, but I needed a woff2, so I tried adding another conversion:

subset-gf-icons % ttx -o '../material-design-icons/variablefont/MaterialSymbolsRounded[FILL,GRAD,opsz,wght]-subset.woff2' --flavor woff2 '../material-design-icons/variablefont/MaterialSymbolsRounded[FILL,GRAD,opsz,wght]-subset.ttf'

Dumping "../material-design-icons/variablefont/MaterialSymbolsRounded[FILL,GRAD,opsz,wght]-subset.ttf" to "../material-design-icons/variablefont/MaterialSymbolsRounded[FILL,GRAD,opsz,wght]-subset.woff2"...
Dumping 'GlyphOrder' table...
Dumping 'head' table...
Dumping 'hhea' table...
Dumping 'maxp' table...
Dumping 'OS/2' table...
Dumping 'hmtx' table...
Dumping 'cmap' table...
Dumping 'prep' table...
Dumping 'loca' table...
Dumping 'glyf' table...
Dumping 'name' table...
Dumping 'post' table...
Dumping 'gasp' table...
Dumping 'GSUB' table...
Dumping 'HVAR' table...
Dumping 'STAT' table...
Dumping 'avar' table...
Dumping 'fvar' table...
Dumping 'gvar' table...

But that generates a 2 MB file for some reason. :-O I tried conversion with an online tool, which seems to work, but only single word symbols are showing. For example

But that might have been lost in the conversion process.

Anyway, this is looking promising, thanks a lot :)

Be the way, is there a way to not show the text in such cases? (when the font is not loaded or the icon not found)

tphinney commented 1 year ago

is there a way to not show the text in such cases? (when the font is not loaded or the icon not found)

Not really. We added having the text because of all the problems with NOT having it that users were experiencing, so it is a feature. But not one that has any easy way to turn off.

If you don’t display the text, you can be missing some icons in a string/group of icons, and not even notice.
If you don’t display the text, and the reason an icon is not displaying is that there is a typo in your text, you can’t see the typo.

So that’s why I decided we needed to have the text in Material Symbols, although we had not had the text characters in Material Icons

EskelCz commented 1 year ago

@tphinney I see. That's a pity, because as a UI designer, I follow the logic that an icon should almost never be used without a text label. You can never be sure that the user interprets the icon correctly. There was a very influential UX article about the user tests for this issue. Therefore I usually use them just helps bring attention to an element and make things more visual. Never as a necessity. I understand that's not what everyone else is doing. :)

Therefore I suffer the opposite issue, which is when the font doesn't load, these texts break my carefully styled layouts. That is even more pressing on mobile where the space is more constrained and the 4MB font file takes ages to load.

It's a good default, but I just wish there was a way to turn it off. Should I open a new issue for it, or is that simply set in stone?

tphinney commented 1 year ago

Hmmm. I see a worse problem even than you are thinking....

when the font doesn't load, these texts break my carefully styled layouts

If the font doesn’t load, you are getting a fallback font, which is guaranteed to have letters for these characters. In that case, my first thought is: it doesn’t even matter what we did in the icon font, and nothing we do in this regard is going to help you. 😢 Hmmm. Why isn’t this what also happens, when the font fails to load and you are using the old Material Icons font? Am I missing something?

If we are talking about behaviors AFTER the font successfully loads, it seems to me that at least in theory, we could have an OpenType layout feature you could turn on, to make the letters non-marking and zero-width... although that might well require also changing the font to no longer declare itself monospaced.

EskelCz commented 1 year ago

@tphinney Before you said you made it that way I was under the impression that this is actually the standard behavior of all icon fonts and was asking for some kind of css workaround. But I didn't mean to derail this conversation with it :) I still need to figure out the WOFF output from the subset-gf-icons. @rsheeter Am I missing something obvious? Sorry I'm new to these python tools.

tphinney commented 1 year ago

My expertise is really limited to the font side; this is not a functionality area I am familiar with.

I would not be shocked if there was some way it could be done, in CSS or otherwise on the site’s side of the fence. But it seems awfully weird/tricky!

rsheeter commented 1 year ago

The production of a 2MB woff2 from a 40KB font makes me think you are getting the xml representation of your subset icon font not an actual woff2.

To make a woff2 you could:

Use FontTools as shown at https://stackoverflow.com/questions/60049960/fonttools-convert-ttf-to-woff2
- You'll need to install the brotli package (https://pypi.org/project/Brotli/)
- You could add woff2 output to subset_gf_icons by adding a dependency on brotli to setup.py and a flag to tell the tool you'd like a woff2
  - Flags should use absl-py as shown in https://github.com/googlefonts/picosvg/blob/main/src/picosvg/picosvg.py
Use woff2_compress as shown at https://rsheeter.github.io/font101/#web-serving

EskelCz commented 1 year ago

@rsheeter Thanks, I managed to convert it while keeping the size down. However the issue with multi-word names persists. I tried the original ttf as well, it's not caused by the conversion. Can you please give it a try? It's really close to being the first working solution.

tphinney commented 1 year ago

Well, I am seeing in the example text that you are missing some of the glyphs for the two-word name. The "c" and "a" are not coming from the icon font. Which would be a reason why the ligature code to get the icon glyph then fails....

(I see the underscore appears to be the one from the icon font, so that is likely not the problem.)

EskelCz commented 1 year ago

@tphinney Interesting, you're correct. It's not multi-word icons, it's all icons containing characters a, b or c. All icons without these letters are working. Thanks for the correction.

@rsheeter Any idea why does subset_gf_icons omit these characters?

For completeness, this is my command:

subset_gf_icons ../material-design-icons/variablefont/MaterialSymbolsRounded[FILL,GRAD,opsz,wght].ttf done chat share close manage_accounts check flag more_vert arrow_back_ios magic_button edit pause play_arrow delete forum star minimize send arrow_drop_down person content_paste location_city interests group verified work cake transgender translate factory wc key_off label_important shield info filter_alt sort groups chevron_left chevron_right

rsheeter commented 1 year ago

Any idea why does subset_gf_icons omit these characters?

My quick hackery assumed zero width space would suffice to break ligatures. I never tested that, it doesn't work. The result was if the sorted unique characters from the icon names you asked to subset activated a ligature it would activate and then some of the icons you wanted to use wouldn't be accessible. Your sample command hits this for the sequence abc.

Should be fixed by https://github.com/rsheeter/subset-gf-icons/pull/2.

EskelCz commented 1 year ago

@rsheeter It works! You're my hero :) Thanks a lot. Reduced from 3MB to 40KB. Well done.

Example for anyone new here, you can subset the variable Google Symbols font into woff2 like so (replace icon names):

pip install -e .
subset_gf_icons ../material-design-icons/variablefont/MaterialSymbolsRounded[FILL,GRAD,opsz,wght].ttf done chat share close check --flavor woff2

MarcoTroost commented 1 year ago

@tphinney Can you please reopen this issue, or should I open a new one about the variable symbol subsetting? I feel like it's a huge waste for such an awesome asset to be basically unusable in production.

Bump

jfbrennan commented 1 year ago

Unless someone beats me to it, I'm going to create a one-pager with complete instructions and examples. I'll link to it here once ready. Thanks a million @rsheeter and @EskelCz

tphinney commented 1 year ago

@tphinney Can you please reopen this issue, or should I open a new one about the variable symbol subsetting? I feel like it's a huge waste for such an awesome asset to be basically unusable in production.

Bump

It was reopened a month ago

MarcoTroost commented 1 year ago

@jfbrennan & @EskelCz Thank you for exploring possible solutions. Using a python script however to convert material symbols from ttf to woff as part of build seems quite impractical in our pipeline.

I really can't grasp why the Google engineers overlooked the quite obvious flaws in distributing Material Symbols as one extremely large, unusable webfont.

Perhaps they could think of a system like the 'ol Icomoon app, where users can select icons & generate a webfont on the fly?

EskelCz commented 1 year ago

@MarcoTroost I was originally looking for a UI solution too, but actually running a script is much more easily automated so I would say it's in fact a better way to go about it. You can also save the list of your icons with the script.

@tphinney @rsheeter Sorry to bother again but after some time of using the subset I noticed a slight flaw in the rendering/antialiasing of the filled variant. It creates minor gaps between the shapes and once you notice it you can't unsee it. :) It's not there with the original font from fonts.googleapis.com, I just checked. Any ideas what might be happening with precision in the export process?

It looks like this, when zoomed in: icons

Tried TTF and it's the same thing.

EDIT: Checked glyph coordinates before and after the subsetting, they look the same. Since they are in integers, it also doesn't look like a precision issue.

Before: GlyphCoordinates([(0, 0),(429, 687),(530, 687),(957, 0),(851, 0),(476, 623),(100, 0),(156, 177),(200, 253),(754, 253),(799, 177)])
After: GlyphCoordinates([(0, 0),(429, 687),(530, 687),(957, 0),(851, 0),(476, 623),(100, 0),(156, 177),(200, 253),(754, 253),(799, 177)])

tphinney commented 1 year ago

Whether this problem happens or not depends on the font rendering engine in use; Apple’s is the biggest problem in this regard. (Note: I am not saying Apple is doing anything terribly wrong! This is a highly unusual glyph rendering situation. It isn’t like a dynamic Fill axis is something one sees every day in a font.)

The original font avoids this Apple-rendering complication by being coded to substitute a completely filled glyph variant when Fill is >99%. Crazy workaround, but effective. So, the subsetting needs to preserve this, too. Currently this is wired up to the 'rclt' OpenType layout feature.

(Not every glyph needs this, but most of them do. That is, all the ones that have an interior fill, similar to the two shown.)

rsheeter commented 1 year ago

The subsetter preserves this sort of feature through layout closure. I turned it off to prevent pulling in excess icons so that's a bug in subset-gf-icons.

rsheeter commented 1 year ago

@tphinney could I impose on you to review https://github.com/rsheeter/subset-gf-icons/pull/3? - looks to me like now if I subset Material Symbols to "star" it additionally retains "star.fill"

tphinney commented 1 year ago

That sounds like the desired behavior! Rod, can you share an output font with me? You’ve got my email etc. :)

behdad commented 1 year ago

To make a woff2 you could:

See https://stackoverflow.com/questions/60049960/fonttools-convert-ttf-to-woff2/77263093#77263093

google / material-design-icons

Subset icon font #1201