502 Server Error: Bad Gateway for url: https://api.github.com/graphql

[simonw]

I'm seeing this intermittent error from the GitHub GraphQL API a lot at the moment, which causes many deploys to fail: https://github.com/simonw/datasette.io/actions/runs/3310172079/jobs/5464078073

  File "/home/runner/work/datasette.io/datasette.io/build_directory.py", line 255, in <module>
    cli()
  File "/opt/hostedtoolcache/Python/3.10.8/x64/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/opt/hostedtoolcache/Python/3.10.8/x64/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/opt/hostedtoolcache/Python/3.10.8/x64/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/hostedtoolcache/Python/3.10.8/x64/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/home/runner/work/datasette.io/datasette.io/build_directory.py", line 138, in cli
    nodes = fetch_plugins(github_token, repos)
  File "/home/runner/work/datasette.io/datasette.io/build_directory.py", line 93, in fetch_plugins
    data = client.execute(
  File "/opt/hostedtoolcache/Python/3.10.8/x64/lib/python3.10/site-packages/python_graphql_client/graphql_client.py", line 54, in execute
    result.raise_for_status()
  File "/opt/hostedtoolcache/Python/3.10.8/x64/lib/python3.10/site-packages/requests/models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 502 Server Error: Bad Gateway for url: https://api.github.com/graphql

I tried dealing with that error previously here:

• 120

[simonw]

I'm going to try breaking up the GIANT GraphQL query into a number of smaller queries.

[simonw]

That didn't fix it: https://github.com/simonw/datasette.io/actions/runs/3322914970/jobs/5492594774

Still the same error. I'll try 20 at a time but I'm not optimistic.

[simonw]

Huh, 20 worked!