search

Munter / subfont

Command line tool to optimize your webfont loading. Aggressive subsetting based on your font use, self-hosting of Google fonts and preloading
MIT License
1.56k stars 29 forks source link

Missing glyph fallback detected #109

Closed Yankovsky closed 4 years ago

Yankovsky commented 4 years ago

In my project I've added link to google fonts: <link href="https://fonts.googleapis.com/css2?family=Roboto+Slab:wght@300;400&family=Roboto:ital,wght@0,300;0,400;0,500;0,700;0,900;1,400&display=swap" rel="stylesheet"/> and process the resulting html using subfont.

Locally on mac everything works great, but during github action execution on ubuntu-latest I've got following warning (more like an error actually):

WARN: Missing glyph fallback detected.
         When your primary webfont doesn't contain the glyphs you use, browsers that don't support unicode-range will load your fallback fonts, which will be a potential waste of bandwidth.
         These glyphs are used on your site, but they don't exist in the font you applied to them:
         - \u{20} ( ) in font-family 'Roboto' (300/normal) at out/ru/course/react/junior/index.html:1:10
         - \u{21} (!) in font-family 'Roboto' (300/normal) at out/ru/course/react/junior/index.html:1:2
         - \u{28} (() in font-family 'Roboto' (300/normal) at out/ru/course/react/junior/index.html:1:223
         - \u{29} ()) in font-family 'Roboto' (300/normal) at out/ru/course/react/junior/index.html:1:242
         - \u{2b} (+) in font-family 'Roboto' (300/normal) at out/ru/course/react/junior/index.html:1:97
...

For some reason it only happens for this particular font. I guess fonts.googleapis.com respond with different css stylesheet depending on OS and environment. Any ideas of what is happening here and how it could be solved?

Yankovsky commented 4 years ago

I can't reliably reproduce this problem.

Yankovsky commented 4 years ago

I am trying to narrow down the issue https://github.com/Yankovsky/subfont-error-demo/runs/846770382?check_suite_focus=true#step:4:14 https://github.com/Yankovsky/subfont-error-demo/tree/b5954bef7a98b904f5e746cb943a113bdb5a2c4d

Yankovsky commented 4 years ago

Hey guys, check this out. [Good request to fonts.google](https://fonts.googleapis.com/css?family=Roboto:300&text=%09%20()%2C-.%2F0123456789%3A%3BACFGIPRSTWYabcdeghijklmnoprstuy~%C2%A0%C2%AB%C2%BB%D0%90%D0%91%D0%92%D0%93%D0%94%D0%95%D0%97%D0%98%D0%99%D0%9A%D0%9B%D0%9C%D0%9D%D0%9E%D0%9F%D0%A0%D0%A1%D0%A2%D0%A3%D0%A4%D0%A5%D0%A6%D0%A7%D0%A8%D0%AB%D0%AC%D0%AF%D0%B0%D0%B1%D0%B2%D0%B3%D0%B4%D0%B5%D0%B6%D0%B7%D0%B8%D0%B9%D0%BA%D0%BB%D0%BC%D0%BD%D0%BE%D0%BF%D1%80%D1%81%D1%82%D1%83%D1%84%D1%85%D1%86%D1%87%D1%88%D1%89%D1%8B%D1%8C%D1%8D%D1%8E%D1%8F%D1%91%E2%80%94%E2%82%BD%E2%84%96&format=woff2) returns one font-face 

@font-face {
  font-family: 'Roboto';
  font-style: normal;
  font-weight: 300;
  src: local('Roboto Light'), local('Roboto-Light'), url(https://fonts.gstatic.com/l/font?kit=KFOlCnqEu92Fr1MmSU5vBh0_IsHAD4pHW1OdLXQxTATpaNU4G3-kkDbkrR1lmSh_gykkuZCZJJKJJ0VcWKLuvPtw_G_2fgDm2wduIaXbqaZQXBsvOh_-6VHzx-Nuwi0LSos_bBQDSSaGp3Mhb97WEMc6vOoI99ctFu7EA_obGDhiiQA9ZBM7gKy6xkPY5fzc5OgDyx9hXnYaLZuuuypcrmpwwLiHFAumHsB1EQawzyAOnqKotiE9_TKUFm5re6QPEq6PCXIMa043LXdd_UvLGo3y2ECoWhMxxdU0Y80NtHOWpg29sGlEZCp96GyGd-CxXfwjpTQBeuj9KMvElhwkUQGeSAXmnuM6V3HfwhlwN4XUpj4XAi-Fa-DidACa_vdtL7daVOC2OXbS8nyH7K_ycV0&skey=11ce8ad5f54705ca&v=v20) format('woff2');
}

but if we add x to list of character when [bad request to google fonts](https://fonts.googleapis.com/css?family=Roboto:300&text=x%09%20()%2C-.%2F0123456789%3A%3BACFGIPRSTWYabcdeghijklmnoprstuy~%C2%A0%C2%AB%C2%BB%D0%90%D0%91%D0%92%D0%93%D0%94%D0%95%D0%97%D0%98%D0%99%D0%9A%D0%9B%D0%9C%D0%9D%D0%9E%D0%9F%D0%A0%D0%A1%D0%A2%D0%A3%D0%A4%D0%A5%D0%A6%D0%A7%D0%A8%D0%AB%D0%AC%D0%AF%D0%B0%D0%B1%D0%B2%D0%B3%D0%B4%D0%B5%D0%B6%D0%B7%D0%B8%D0%B9%D0%BA%D0%BB%D0%BC%D0%BD%D0%BE%D0%BF%D1%80%D1%81%D1%82%D1%83%D1%84%D1%85%D1%86%D1%87%D1%88%D1%89%D1%8B%D1%8C%D1%8D%D1%8E%D1%8F%D1%91%E2%80%94%E2%82%BD%E2%84%96&format=woff2) returns full list of font faces:

/* cyrillic-ext */
@font-face {
  font-family: 'Roboto';
  font-style: normal;
  font-weight: 300;
  src: local('Roboto Light'), local('Roboto-Light'), url(https://fonts.gstatic.com/s/roboto/v20/KFOlCnqEu92Fr1MmSU5fCRc4AMP6lbBP.woff2) format('woff2');
  unicode-range: U+0460-052F, U+1C80-1C88, U+20B4, U+2DE0-2DFF, U+A640-A69F, U+FE2E-FE2F;
}
/* cyrillic */
@font-face {
  font-family: 'Roboto';
  font-style: normal;
  font-weight: 300;
  src: local('Roboto Light'), local('Roboto-Light'), url(https://fonts.gstatic.com/s/roboto/v20/KFOlCnqEu92Fr1MmSU5fABc4AMP6lbBP.woff2) format('woff2');
  unicode-range: U+0400-045F, U+0490-0491, U+04B0-04B1, U+2116;
}
/* greek-ext */
@font-face {
  font-family: 'Roboto';
  font-style: normal;
  font-weight: 300;
  src: local('Roboto Light'), local('Roboto-Light'), url(https://fonts.gstatic.com/s/roboto/v20/KFOlCnqEu92Fr1MmSU5fCBc4AMP6lbBP.woff2) format('woff2');
  unicode-range: U+1F00-1FFF;
}
/* greek */
@font-face {
  font-family: 'Roboto';
  font-style: normal;
  font-weight: 300;
  src: local('Roboto Light'), local('Roboto-Light'), url(https://fonts.gstatic.com/s/roboto/v20/KFOlCnqEu92Fr1MmSU5fBxc4AMP6lbBP.woff2) format('woff2');
  unicode-range: U+0370-03FF;
}
/* vietnamese */
@font-face {
  font-family: 'Roboto';
  font-style: normal;
  font-weight: 300;
  src: local('Roboto Light'), local('Roboto-Light'), url(https://fonts.gstatic.com/s/roboto/v20/KFOlCnqEu92Fr1MmSU5fCxc4AMP6lbBP.woff2) format('woff2');
  unicode-range: U+0102-0103, U+0110-0111, U+0128-0129, U+0168-0169, U+01A0-01A1, U+01AF-01B0, U+1EA0-1EF9, U+20AB;
}
/* latin-ext */
@font-face {
  font-family: 'Roboto';
  font-style: normal;
  font-weight: 300;
  src: local('Roboto Light'), local('Roboto-Light'), url(https://fonts.gstatic.com/s/roboto/v20/KFOlCnqEu92Fr1MmSU5fChc4AMP6lbBP.woff2) format('woff2');
  unicode-range: U+0100-024F, U+0259, U+1E00-1EFF, U+2020, U+20A0-20AB, U+20AD-20CF, U+2113, U+2C60-2C7F, U+A720-A7FF;
}
/* latin */
@font-face {
  font-family: 'Roboto';
  font-style: normal;
  font-weight: 300;
  src: local('Roboto Light'), local('Roboto-Light'), url(https://fonts.gstatic.com/s/roboto/v20/KFOlCnqEu92Fr1MmSU5fBBc4AMP6lQ.woff2) format('woff2');
  unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+2000-206F, U+2074, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD;
}

Later in subfont code there is a line

        const characterSet = fontkit.create(Object.values(fontUsage.subsets)[0])
          .characterSet;

and here with bad request we get only cyrillic-ext subset hence the error.

Yankovsky commented 4 years ago

Do you know if fonts.googleapis is open sourced?

Yankovsky commented 4 years ago

Probably easier solution for now is to use local fonts.

Yankovsky commented 4 years ago

If requests are made from server (no user agent) when they return

@font-face {
  font-family: 'Roboto';
  font-style: normal;
  font-weight: 300;
  src: local('Roboto Light'), local('Roboto-Light'), url(https://fonts.gstatic.com/l/font?kit=KFOlCnqEu92Fr1MmSU5vAwWnKMjSvaFiVlWbMWgtUADtbNE0F3Oo6UyY0RV2jTxihCIWj6SkIJeCKkpQXKbquOdv4XTzdQ1QDry7Ib7ahqdkXDsvHh_16Uzz1ONnxRMMdosobAkDQya0p0EgTN_WEPAzl-M469AxEO7VA-UbGjhgiQY9YhM4gKG6yEPa5f7c4ugFyx1hXHYkLaWuuSpermxwxriFFAmmEMB5EQKwyyASnryotCE__TSUEG5pe6YPHK6BCXAMaU4xLXFd_0vJG7Pz5kCqWhExw9UyY88NtnOYpgO9s2lBZCx96myEd_-xTP4wpzoBaej_KMnEkhwgUQqeQQXkntk6bXHdwhtwPIXDpiIXBi-Ba-zieDKaiq82E_EBeYbXFwOyzwrh-A&skey=11ce8ad5f54705ca&v=v20) format('truetype');
}

and

@font-face {
  font-family: 'Roboto';
  font-style: normal;
  font-weight: 300;
  src: local('Roboto Light'), local('Roboto-Light'), url(https://fonts.gstatic.com/s/roboto/v20/KFOlCnqEu92Fr1MmSU5fBBc9.ttf) format('truetype');
}

respectively

Yankovsky commented 4 years ago

Found the difference between local and github CI. I had pyftsubset installed locally. Thats why condition here were true https://github.com/Munter/subfont/blob/8b1d0a153dbefcbb68f2441adc4aefff4b77ce20/lib/subsetFonts.js#L303

Yankovsky commented 4 years ago

Can you help me to figure out what local means in that context? I thought that it was about working with local font files, but here I use google fonts link.

Yankovsky commented 4 years ago

I've prepared a repo with minimal code to reproduce the problem https://github.com/Yankovsky/subfont-error-demo

papandreou commented 4 years ago

but if we add x to list of character when bad request to google fonts returns full list of font faces:

Interesting finding!

Seems like fonts.googleapis.com/css?text=... switches to a mode where you get a further subdivision of the fonts when you cross some sort of threshold. It doesn't seem to happen with the User-Agent string that subfont uses, though:

$ curl -H 'User-Agent: AssetGraph v6.1.1' 'https://fonts.googleapis.com/css?family=Roboto:300&text=x%09%20()%2C-.%2F0123456789%3A%3BACFGIPRSTWYabcdeghijklmnoprstuy~%C2%A0%C2%AB%C2%BB%D0%90%D0%91%D0%92%D0%93%D0%94%D0%95%D0%97%D0%98%D0%99%D0%9A%D0%9B%D0%9C%D0%9D%D0%9E%D0%9F%D0%A0%D0%A1%D0%A2%D0%A3%D0%A4%D0%A5%D0%A6%D0%A7%D0%A8%D0%AB%D0%AC%D0%AF%D0%B0%D0%B1%D0%B2%D0%B3%D0%B4%D0%B5%D0%B6%D0%B7%D0%B8%D0%B9%D0%BA%D0%BB%D0%BC%D0%BD%D0%BE%D0%BF%D1%80%D1%81%D1%82%D1%83%D1%84%D1%85%D1%86%D1%87%D1%88%D1%89%D1%8B%D1%8C%D1%8D%D1%8E%D1%8F%D1%91%E2%80%94%E2%82%BD%E2%84%96'
@font-face {
  font-family: 'Roboto';
  font-style: normal;
  font-weight: 300;
  src: local('Roboto Light'), local('Roboto-Light'), url(https://fonts.gstatic.com/s/roboto/v20/KFOlCnqEu92Fr1MmSU5fBBc9.ttf) format('truetype');
}
papandreou commented 4 years ago

Ah, we override that when we try to get a woff or woff2 out, and the woff2 one results in those split up @font-face declarations being returned.

papandreou commented 4 years ago

Working on a fix here: https://github.com/Munter/subfont/pull/115

Yankovsky commented 4 years ago

@papandreou thanks for your help! Can you tell me where this User-Agent: AssetGraph v6.1.1 is coming from?

papandreou commented 4 years ago

Can you tell me where this User-Agent: AssetGraph v6.1.1 is coming from?

It comes from here: https://github.com/assetgraph/assetgraph/blob/d4026e3c030c010c8db09c26c3d32259bfe02f5f/lib/AssetGraph.js#L70-L72

Yankovsky commented 4 years ago

Got it. I guess if fonts.google.com can't detect browser, it will just send ttf file. So the result for User-Agent: AssetGraph v6.1.1 and no User-Agent is the same.

papandreou commented 4 years ago

I guess if fonts.google.com can't detect browser, it will just send ttf file. So the result for User-Agent: AssetGraph v6.1.1 and no User-Agent is the same.

Yes, correct! :+1: