previously, if docker registry check failed, no result would be cached, but then the unset None value would be returned from at_most_every.
Fix: turn errors to false before caching results in at_most_every
Secondary fix: ensure errors don't result in spurious cache hits returning unset last_result
Encountered trying to migrate docker registry to Artifact Registry, where I don't have the credentials right yet, so the first docker registry check failed with 403. After that, health checks fail with:
binder-7f598cc5d8-br4g5 binder [E 230425 07:43:29 web:1798] Uncaught exception GET /health (10.4.1.132)
binder-7f598cc5d8-br4g5 binder HTTPServerRequest(protocol='https', host='gke2.staging.mybinder.org', method='GET', uri='/health', version='HTTP/1.1', remote_ip='10.4.1.132')
binder-7f598cc5d8-br4g5 binder Traceback (most recent call last):
binder-7f598cc5d8-br4g5 binder File "/usr/local/lib/python3.11/site-packages/tornado/web.py", line 1713, in _execute
binder-7f598cc5d8-br4g5 binder result = await result
binder-7f598cc5d8-br4g5 binder ^^^^^^^^^^^^
binder-7f598cc5d8-br4g5 binder File "/usr/local/lib/python3.11/site-packages/binderhub/health.py", line 169, in get
binder-7f598cc5d8-br4g5 binder overall, checks = await self.check_all()
binder-7f598cc5d8-br4g5 binder ^^^^^^^^^^^^^^^^^^^^^^
binder-7f598cc5d8-br4g5 binder File "/usr/local/lib/python3.11/site-packages/binderhub/health.py", line 159, in check_all
binder-7f598cc5d8-br4g5 binder results.append(dict({"service": service}, **result))
binder-7f598cc5d8-br4g5 binder ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
binder-7f598cc5d8-br4g5 binder TypeError: dict() argument after ** must be a mapping, not NoneType
previously, if docker registry check failed, no result would be cached, but then the unset
None
value would be returned from at_most_every.Encountered trying to migrate docker registry to Artifact Registry, where I don't have the credentials right yet, so the first docker registry check failed with 403. After that, health checks fail with: