Google Fonts API key invalid?

[N-Wouda]

Hi! A few hours ago our doc builds over at PyVRP started to fail, with this error message:

>>>make html --directory=docs
make: Entering directory '/home/runner/work/PyVRP/PyVRP/docs'
Running Sphinx v7.2.6
Copying example notebooks into docs/source/examples/
making output directory... done
loading intersphinx inventory from https://docs.python.org/3/objects.inv...
loading intersphinx inventory from https://www.sphinx-doc.org/en/master/objects.inv...
loading intersphinx inventory from https://numpy.org/doc/stable/objects.inv...
loading intersphinx inventory from https://matplotlib.org/stable/objects.inv...
Fetching: https://content-webfonts.googleapis.com/v1/webfonts?key=AIzaSyAa8yy0GdcGPHdtD083HiGGx_S0vMPScDM with {'x-referer': 'https://explorer.apis.google.com'}
Extension error (sphinx_immaterial.google_fonts):
Handler <function _builder_inited at 0x7fc4f79f7b00> for event 'builder-inited' threw an exception (exception: 403 Client Error: Forbidden for url: https://content-webfonts.googleapis.com/v1/webfonts?key=AIzaSyAa8yy0GdcGPHdtD083HiGGx_S0vMPScDM)
make: Leaving directory '/home/runner/work/PyVRP/PyVRP/docs'
make: *** [Makefile:[20](https://github.com/PyVRP/PyVRP/actions/runs/8191559344/job/22405463658#step:10:21): html] Error 2
Error: Process completed with exit code 2.

See this build for more details. We are using the latest sphinx-immaterial release (0.11.10), and all this worked until a few hours ago.

When I follow the link to the font over at Google myself, I get the following:

Which now has an error code of 400, not 403 as in the error message I see in our build. I'm unsure if that is the same issue. Do you know what's going on?

[2bndy5]

Fetching: https://content-webfonts.googleapis.com/v1/webfonts?key=AIzaSyAa8yy0GdcGPHdtD083HiGGx_S0vMPScDM with {'x-referer': 'https://explorer.apis.google.com'}

Following the link in your browser is not the same as the HTTP request being made in the source code. Here we see that the HTTP request headers include {'x-referer': 'https://explorer.apis.google.com'}: https://github.com/jbms/sphinx-immaterial/blob/972eafea3e3488899558b394720e15af7537f348/sphinx_immaterial/google_fonts.py#L147-L151

>>> import requests
>>> r = requests.get("https://content-webfonts.googleapis.com/v1/webfonts?key=AIzaSyAa8yy0GdcGPHdtD083HiGGx_S0vMPScDM", headers={'x-referer': 'https://explorer.apis.google.com'})
>>> import json
>>> print(json.dumps(r.json(), indent=2))
{
  "error": {
    "code": 403,
    "message": "Requests to this API webfonts method google.fonts.v1.WebFontsService.ListWebFonts are blocked.",
    "errors": [
      {
        "message": "Requests to this API webfonts method google.fonts.v1.WebFontsService.ListWebFonts are blocked.",
        "domain": "global",
        "reason": "forbidden"
      }
    ],
    "status": "PERMISSION_DENIED",
    "details": [
      {
        "@type": "type.googleapis.com/google.rpc.ErrorInfo",
        "reason": "API_KEY_SERVICE_BLOCKED",
        "domain": "googleapis.com",
        "metadata": {
          "service": "webfonts.googleapis.com",
          "consumer": "projects/292824132082"
        }
      }
    ]
  }
}

@jbms It does look like the API key is being denied. IIRC, this key was extracted from the google fonts site (somehow).

[peske]

Same here. Log:

Fetching: https://content-webfonts.googleapis.com/v1/webfonts?key=AIzaSyAa8yy0GdcGPHdtD083HiGGx_S0vMPScDM with {'x-referer': 'https://explorer.apis.google.com'/}
Extension error (sphinx_immaterial.google_fonts):
Handler <function _builder_inited at 0x7f4d6c950b80> for event 'builder-inited' threw an exception (exception: 403 Client Error: Forbidden for url: https://content-webfonts.googleapis.com/v1/webfonts?key=AIzaSyAa8yy0GdcGPHdtD083HiGGx_S0vMPScDM)

[2bndy5]

If you have a Google Fonts API key, then you could try monkeypatching the theme source to use it as a workflow secret variable via an environment variable.

# in conf.py

from os import environ
import sphinx_immaterial.google_fonts

if "MY_SECRET_ENV_VAR_NAME" in environ:
    google_fonts._GOOGLE_FONTS_API_KEY = environ["MY_SECRET_ENV_VAR_NAME"]

# in docs build CI workflow (.yml file)

      - name: run sphinx-build
        env:
          MY_SECRET_ENV_VAR_NAME: ${{ secrets.MY_GOOGLE_FONTS_API_KEY }}
        run: sphinx-build docs docs/_build/html -E -W

I don't know if distributing a Google Fonts API key in the theme source is a good idea because anyone can copy/use the key which supposed to be specific to a Google account.

---

Personally, I would prefer switching to FontSource REST API for downloading fonts.

[2bndy5]

Looking at upstream's privacy plugin (the mechanism that caches google font files for mkdocs-material builds), I see that the source code will

1. scan the rendered HTML files
2. download any links found and save to a cache folder
3. replace the links found with paths to cached files

In the case of Google Fonts, the URL (after rendering the HTML template with default font settings) is https://fonts.googleapis.com/css?family=Roboto:300,300i,400,400i,700,700i%7CRoboto+Mono:400,400i,700,700i&display=fallback. In that result, you'll find the URLs needed to download the fonts as well (ie https://fonts.gstatic.com/s/robotomono/v23/L0xTDF4xlVMF-BfR8bXMIhJHg45mwgGEFl0_3vrtSM1J-gEPT5Ese6hmHSh0mQ.woff2)

Luckily, no API key is needed for upstream's approach.

[2bndy5]

FYI, using cached fonts (via sphinx_immaterial_external_resource_cache_dir) checked into docs' source does not work.

[jbms]

Looking at the current Google Fonts website, it seems to fetch the font information from: https://fonts.google.com/metadata/fonts

You can fetch that without any API key or referrer and it returns the metadata in JSON. The format is not the same as the existing metadata we were fetching, though, so we will need to make some adjustments.

[2bndy5]

The format is not the same as the existing metadata we were fetching, though, so we will need to make some adjustments.

I see also that we can download fonts directly from the github repo (as raw blobs), but the repo organizes font files by license type (ofl or ufl).

the JSON data

```json { "family": "Roboto", "displayName": null, "category": "Sans Serif", "stroke": "Sans Serif", "classifications": [], "size": 173710, "subsets": [ "menu", "cyrillic", "cyrillic-ext", "greek", "greek-ext", "latin", "latin-ext", "vietnamese" ], "fonts": { "100": { "thickness": 2, "slant": 1, "width": 7, "lineHeight": 1.171875 }, "100i": { "thickness": 2, "slant": 4, "width": 7, "lineHeight": 1.171875 }, "300": { "thickness": 4, "slant": 1, "width": 7, "lineHeight": 1.171875 }, "300i": { "thickness": 4, "slant": 4, "width": 7, "lineHeight": 1.171875 }, "400": { "thickness": 5, "slant": 1, "width": 7, "lineHeight": 1.171875 }, "400i": { "thickness": 5, "slant": 4, "width": 7, "lineHeight": 1.171875 }, "500": { "thickness": 6, "slant": 1, "width": 7, "lineHeight": 1.171875 }, "500i": { "thickness": 6, "slant": 4, "width": 7, "lineHeight": 1.171875 }, "700": { "thickness": 6, "slant": 1, "width": 7, "lineHeight": 1.171875 }, "700i": { "thickness": 6, "slant": 4, "width": 7, "lineHeight": 1.171875 }, "900": { "thickness": 7, "slant": 1, "width": 7, "lineHeight": 1.171875 }, "900i": { "thickness": 7, "slant": 4, "width": 7, "lineHeight": 1.171875 } }, "axes": [], "designers": [ "Christian Robertson" ], "lastModified": "2022-09-21", "dateAdded": "2013-01-09", "popularity": 2, "trending": 1058, "defaultSort": 1, "androidFragment": null, "isNoto": false, "colorCapabilities": [], "primaryScript": "", "primaryLanguage": "" }, "family": "Roboto Mono", "displayName": null, "category": "Monospace", "stroke": null, "classifications": [ "Monospace" ], "size": 191258, "subsets": [ "menu", "cyrillic", "cyrillic-ext", "greek", "latin", "latin-ext", "vietnamese" ], "fonts": { "100": { "thickness": null, "slant": null, "width": null, "lineHeight": 1.31884765625 }, "100i": { "thickness": null, "slant": null, "width": null, "lineHeight": 1.31884765625 }, "200": { "thickness": null, "slant": null, "width": null, "lineHeight": 1.31884765625 }, "200i": { "thickness": null, "slant": null, "width": null, "lineHeight": 1.31884765625 }, "300": { "thickness": null, "slant": null, "width": null, "lineHeight": 1.31884765625 }, "300i": { "thickness": null, "slant": null, "width": null, "lineHeight": 1.31884765625 }, "400": { "thickness": null, "slant": null, "width": null, "lineHeight": 1.31884765625 }, "400i": { "thickness": null, "slant": null, "width": null, "lineHeight": 1.31884765625 }, "500": { "thickness": null, "slant": null, "width": null, "lineHeight": 1.31884765625 }, "500i": { "thickness": null, "slant": null, "width": null, "lineHeight": 1.31884765625 }, "600": { "thickness": null, "slant": null, "width": null, "lineHeight": 1.31884765625 }, "600i": { "thickness": null, "slant": null, "width": null, "lineHeight": 1.31884765625 }, "700": { "thickness": null, "slant": null, "width": null, "lineHeight": 1.31884765625 }, "700i": { "thickness": null, "slant": null, "width": null, "lineHeight": 1.31884765625 } }, "axes": [ { "tag": "wght", "min": 100.0, "max": 700.0, "defaultValue": 400.0 } ], "designers": [ "Christian Robertson" ], "lastModified": "2023-09-14", "dateAdded": "2015-05-13", "popularity": 15, "trending": 931, "defaultSort": 15, "androidFragment": null, "isNoto": false, "colorCapabilities": [], "primaryScript": "", "primaryLanguage": "" } ```

does not include the ofl/ufl license specification, so we can't download raw blobs from github...

[jbms]

Looking at upstream's privacy plugin (the mechanism that caches google font files for mkdocs-material builds), I see that the source code will

1. scan the rendered HTML files
2. download any links found and save to a cache folder
3. replace the links found with paths to cached files

In the case of Google Fonts, the URL (after rendering the HTML template with default font settings) is https://fonts.googleapis.com/css?family=Roboto:300,300i,400,400i,700,700i%7CRoboto+Mono:400,400i,700,700i&display=fallback. In that result, you'll find the URLs needed to download the fonts as well (ie https://fonts.gstatic.com/s/robotomono/v23/L0xTDF4xlVMF-BfR8bXMIhJHg45mwgGEFl0_3vrtSM1J-gEPT5Ese6hmHSh0mQ.woff2)

Luckily, no API key is needed for upstream's approach.

In our existing approach, the API key is just needed for fetching the overall font metadata, i.e. the list of variants available for each font.

Fetching the actual font data is done using the same HTTP requests that the browser makes when visiting a website that uses Google fonts in the normal way, by including them in their website, and no API keys are needed.

The mkdocs-material approach is to just hard code a list of variants to fetch. That is initially what I implemented as well, but it is a bit annoying because the variants that are available differ depending on the font family, so really the user needs to be able to specify the variants to fetch for each font. To avoid that problem, I added in the fetching of the metadata so that we could download all variants for each font that is used.

It also wouldn't be a problem to obtain a Google Fonts API for this project and embed it in the source code as we have done with the existing API key --- I could do that. But just using the other metadata URL which requires no API key would seem to be simpler.

[2bndy5]

I also don't see available "style" (aka variants) listed in the suggested JSON. It is more implicit (with i suffix for italic fonts) in the listed family's fonts field:

        "400": {
          "thickness": 5,
          "slant": 1,
          "width": 7,
          "lineHeight": 1.171875
        },
        "400i": {
          "thickness": 5,
          "slant": 4,
          "width": 7,
          "lineHeight": 1.171875
        },

This means

• bold: 700
• regular: 400
• thin: 100
• bold italic: 700i
• regular italic: 400i
• thin italic: 100i

[!CAUTION] Not all fonts have the appropriate weights for associated styles. For example, ABeeZee only has a regular (400) and italic (400i) style, no bold or thin weights available.

[2bndy5]

branch created at no-google-fonts-api-key

Although I think this means users cannot specify the font style in conf.py.

[2bndy5]

Although I think this means users cannot specify the font style in conf.py.

This doesn't seem to be supported upstream either. I don't know why I was concerned. PR incoming.

[peske]

Thanks @2bndy5!

Do you know will a new release (0.11.11) be created soon or not?

[2bndy5]

building and publishing as I type this...

[N-Wouda]

Fantastic, thank you @2bndy5 for the fast solution!

[2bndy5]

fix released in v0.11.11

[2bndy5]

Thanks to @jbms for the metadata lead. I was headed in the wrong direction without it. 🥇