jupyterhub / binderhub

Run your code in the cloud, with technology so advanced, it feels like magic!
https://binderhub.readthedocs.io
BSD 3-Clause "New" or "Revised" License
2.54k stars 388 forks source link

catch errors before caching in binderhub health check #1662

Closed minrk closed 1 year ago

minrk commented 1 year ago

previously, if docker registry check failed, no result would be cached, but then the unset None value would be returned from at_most_every.

Encountered trying to migrate docker registry to Artifact Registry, where I don't have the credentials right yet, so the first docker registry check failed with 403. After that, health checks fail with:

binder-7f598cc5d8-br4g5 binder [E 230425 07:43:29 web:1798] Uncaught exception GET /health (10.4.1.132)
binder-7f598cc5d8-br4g5 binder     HTTPServerRequest(protocol='https', host='gke2.staging.mybinder.org', method='GET', uri='/health', version='HTTP/1.1', remote_ip='10.4.1.132')
binder-7f598cc5d8-br4g5 binder     Traceback (most recent call last):
binder-7f598cc5d8-br4g5 binder       File "/usr/local/lib/python3.11/site-packages/tornado/web.py", line 1713, in _execute
binder-7f598cc5d8-br4g5 binder         result = await result
binder-7f598cc5d8-br4g5 binder                  ^^^^^^^^^^^^
binder-7f598cc5d8-br4g5 binder       File "/usr/local/lib/python3.11/site-packages/binderhub/health.py", line 169, in get
binder-7f598cc5d8-br4g5 binder         overall, checks = await self.check_all()
binder-7f598cc5d8-br4g5 binder                           ^^^^^^^^^^^^^^^^^^^^^^
binder-7f598cc5d8-br4g5 binder       File "/usr/local/lib/python3.11/site-packages/binderhub/health.py", line 159, in check_all
binder-7f598cc5d8-br4g5 binder         results.append(dict({"service": service}, **result))
binder-7f598cc5d8-br4g5 binder                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
binder-7f598cc5d8-br4g5 binder     TypeError: dict() argument after ** must be a mapping, not NoneType