google / fonts

Font files available from Google Fonts, and a public issue tracker for all things Google Fonts
https://fonts.google.com
18.2k stars 2.62k forks source link

CSS API Breaking Changes #3870

Closed ayuhito closed 2 years ago

ayuhito commented 3 years ago

Hiya,

A recent update to the CSS API essentially broke a portion of the Fontsource library with no option for me to actually fix the bug since the data previously used is no longer served by the platform.

Fontsource uses BOTH the CSS APIv1 and CSS APIv2 to gather all the files and information it needs since both APIs serve different files and formats which therefore can be offered as additional flexibility to developers downstream.

It seems that the CSS APIv1 was changed to serve files with the unicode-range CSS selector in the recent Noto update, a feature previously unique to CSS APIv2.

Before:

...
/* japanese */
@font-face {
  font-family: 'Noto Sans JP';
  font-style: normal;
  font-weight: 400;
  src: url(https://fonts.gstatic.com/s/notosansjp/v28/-F62fjtqLzI2JPCgQBnw7HFYyQgP.woff2) format('woff2');
}
...

After (mirrors CSS APIv2):

...
/* [2] */
@font-face {
  font-family: 'Noto Sans JP';
  font-style: normal;
  font-weight: 400;
  src: url(https://fonts.gstatic.com/s/notosansjp/v36/-F62fjtqLzI2JPCgQBnw7HFow2oe2EcP5pp0erwTqsSWs9Jezazjcb4.2.woff2) format('woff2');
  unicode-range: U+ffd7, U+ffda-ffdc, U+ffe0-ffe2, U+ffe4, U+ffe6, U+ffe8-ffee, U+1f100-1f10c, U+1f110-1f16c, U+1f170-1f1ac, U+1f200-1f202, U+1f210-1f234;
}
...

At least, my expectation would be that CSS APIv1 should be untouched while CSS APIv2 would be getting new updates? I would assume that's why we have two API versions in the first place. Would the team at Google be open to considering reverting the changes made to the CSS APIv1 or perhaps offer any other alternative solutions that we could collaborate on with?

References: fontsource/fontsource#353

rsheeter commented 3 years ago

We do not guarrantee the stability of the CSS content. For context, the v1 api has long supported unicode-range for most fonts and has repeatedly changed it's response over time for woff2, unicode-range, font-display, and to work around various quirks in browser/os capability.

Can you tell us a little more about your use case? It sounds like maybe you are using the CSS API to acquire metadata about the library?

ayuhito commented 3 years ago

That is correct, we do scrape the CSS API's using google-font-metadata since the Developer API doesn't give us enough metadata to work with. Primarily, what we are looking for is the download links to the font files which we can then pull for self-hosting purposes.

Please CTRL+F to "noto-sans-jp" to see the differences easily. The main difference is that previously CSS 1 was split into "latin" and "japanese" subsets whilst CSS 2 was split into many different smaller numbered subsets. Both formats have different usecases for developers, though I never really encouraged the former.

However, it seems that CSS 1 had changes done to it and now it does the unicode subsetting, as shown in my previous post with the CSS excerpts.

I am more than well aware that the stability of the CSS content cannot be guaranteed, but I would highly appreciate it if we could work towards a more stable solution for this since Fontsource does get millions of NPM downloads monthly (and growing rapidly) and therefore there are many developers affected by this. I'd be more than happy to discuss and cooperate with the Google team to any capacity that is reasonable.

rsheeter commented 3 years ago

It's on our radar that we need to improve the metadata APIs exposed. The example of the metadata you are creating, which is presumably what you want/need, is very helpful. Unfortunately it will take us some time to get to as we're a relatively small team and this has to compete for priority with everything else.

I'd be more than happy to discuss and cooperate with the Google team to any capacity that is reasonable.

Most appreciated. I wonder if I could impose on you to file a separate issue that describes the metadata api(s) that you'd like to have and ping me on it?

what we are looking for is the download links to the font files which we can then pull for self-hosting purposes.

Short term if you have something like a batch job that builds up that metadata and you can control the user agent there then you can cherry-pick ones that get the file format you want. Undocumented and not guarranteed to continue to work, use at own risk :)

# raw font file
$ curl 'https://fonts.googleapis.com/css?family=Noto+Sans+JP&subset=japanese'
@font-face {
  font-family: 'Noto Sans JP';
  font-style: normal;
  font-weight: 400;
  src: url(https://fonts.gstatic.com/s/notosansjp/v36/-F62fjtqLzI2JPCgQBnw7HFYyQgM.otf) format('opentype');
}

# woff
curl 'https://fonts.googleapis.com/css?family=Noto+Sans+JP&subset=japanese' -H 'User-Agent: Mozilla/5.0 (Windows NT 6.3; Trident/7.0; rv:11.0) like Gecko'
@font-face {
  font-family: 'Noto Sans JP';
  font-style: normal;
  font-weight: 400;
  src: url(https://fonts.gstatic.com/s/notosansjp/v36/-F62fjtqLzI2JPCgQBnw7HFYyQgJ.woff) format('woff');
}

# woff2
curl 'https://fonts.googleapis.com/css?family=Noto+Sans+JP&subset=japanese' -H 'User-Agent: Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Safari/537.36 Edge/15.10130'
@font-face {
  font-family: 'Noto Sans JP';
  font-style: normal;
  font-weight: 400;
  src: url(https://fonts.gstatic.com/s/notosansjp/v36/-F62fjtqLzI2JPCgQBnw7HFYyQgP.woff2) format('woff2');
}

I would think we're less likely to change behavior for old browsers than new ones.

ayuhito commented 3 years ago

It's on our radar that we need to improve the metadata APIs exposed. The example of the metadata you are creating, which is presumably what you want/need, is very helpful. Unfortunately it will take us some time to get to as we're a relatively small team and this has to compete for priority with everything else.

Of course, I understand. Just to present our side of the story to help you guys decide on priorities, although these Developer APIs do have a small number of projects oriented around them, the dependents of those specific projects may have large numbers yet sometimes is plagued with unreliability unfitting for them due to the haphazard ways we get around not having a comprehensive enough API.

e.g. Fontsource or majodev/google-webfonts-helper#131 and so on...

Most appreciated. I wonder if I could impose on you to file a separate issue that describes the metadata api(s) that you'd like to have and ping me on it?

Likewise, I appreciate the open dialogue. I think an ideal API may be something similar to the Fontsource API, where I re-export google-font-metadata's data. It's essentially what I would have wanted from Google, however, at least your team will have more flexibility being the source of all the metadata.

I'll be more than happy to create issues for the API spec of this when I get time, and also an additional Variable Fonts API since Google's repertoire of variable fonts have significantly increased. We have to scrape the Variable Fonts page currently, although previously there have been silent design changes that have required reactive changes to the project to keep things working - not ideal.

Short term if you have something like a batch job that builds up that metadata and you can control the user agent there then you can cherry-pick ones that get the file format you want. Undocumented and not guarranteed to continue to work, use at own risk :)

Thank you, I think we can work with that as a temporary measure. :)

It'd be amazing to maybe get a heads-up if that undocumented service is planned on getting taken down for us to take appropriate measures since the Fontsource project auto-publishes any package changes without review. Communication would save a lot of headaches down the line to prevent cases like this sprouting again.

ayuhito commented 2 years ago

I'll close this since #3880 suggests a long term solution. Thanks once again!