Kube-janitor doesn't work at all if one of the API's is inaccessible

[bdevnani3]

When running kube-janitor on our cluster, we got 503's when attempting to access the api metrics/v1alpha1.

2019-08-12 05:06:00,534 DEBUG: Collecting resources for metrics/v1alpha1..
2019-08-12 05:06:00,734 DEBUG: https://172.20.0.1:443 "GET /apis/metrics/v1alpha1/ HTTP/1.1" 503 20
2019-08-12 05:06:00,734 ERROR: Failed to clean up: 503 Server Error: Service Unavailable for url: https://172.20.0.1:443/apis/metrics/v1alpha1/
Traceback (most recent call last):
  File "/kube_janitor/main.py", line 51, in run_loop
    dry_run=dry_run)
  File "/kube_janitor/janitor.py", line 215, in clean_up
    for _type in resource_types:
  File "/kube_janitor/resources.py", line 39, in get_namespaced_resource_types
    for api_version, resource in discover_namespaced_api_resources(api):
  File "/kube_janitor/resources.py", line 32, in discover_namespaced_api_resources
    r2.raise_for_status()
  File "/usr/local/lib/python3.7/site-packages/requests/models.py", line 940, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 503 Server Error: Service Unavailable for url: https://172.20.0.1:443/apis/metrics/v1alpha1/

This prevented kube-janitor from running at all on our cluster.

Is it by design supposed to fail when one API is inaccessible?

In addition(and I think this has been raised in another issue), this would also be avoided in our specific use-case if we didn't iterate through all but instead just the required APIs(That, of course, doesn't address the root cause).

[hjacobs]

Good point, the API discovery should be more robust.

[bdevnani3]

I can work on a push that attempts to fix this if that's okay?

[hjacobs]

Yes, a PR is welcome :smile:

[csko]

I believe this is fixed in #42.