Exporter tries to get metrics without logging on after HMC restart

[andy-maier]

From a client:

# tail -20  zhmc_prom_exporter.log
    raise new_exc  # AuthError
zhmc_prometheus_exporter.zhmc_prometheus_exporter.AuthError: Authentication error returned from the HMC at 10.174.38.89 using userid 'zhmcexporter' defined in HMC credentials file /etc/zhmc-prometheus-exporter/hmccreds.yaml: HTTP authentication failed: No session id was provided (must login) (HMC operation GET /api/services/metrics/context/1, HTTP status 403.4)
Traceback (most recent call last):
  File "/usr/lib64/python3.9/wsgiref/handlers.py", line 137, in run
    self.result = application(self.environ, self.start_response)
  File "/usr/local/lib/python3.9/site-packages/prometheus_client/exposition.py", line 129, in prometheus_app
    status, headers, output = _bake_output(registry, accept_header, accept_encoding_header, params, disable_compression)
  File "/usr/local/lib/python3.9/site-packages/prometheus_client/exposition.py", line 105, in _bake_output
    output = encoder(registry)
  File "/usr/local/lib/python3.9/site-packages/prometheus_client/openmetrics/exposition.py", line 21, in generate_latest
    for metric in registry.collect():
  File "/usr/local/lib/python3.9/site-packages/prometheus_client/registry.py", line 97, in collect
    yield from collector.collect()
  File "/usr/local/lib/python3.9/site-packages/zhmc_prometheus_exporter/zhmc_prometheus_exporter.py", line 1343, in collect
    break
  File "/usr/lib64/python3.9/contextlib.py", line 137, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/usr/local/lib/python3.9/site-packages/zhmc_prometheus_exporter/zhmc_prometheus_exporter.py", line 191, in zhmc_exceptions
    raise new_exc  # AuthError
zhmc_prometheus_exporter.zhmc_prometheus_exporter.AuthError: Authentication error returned from the HMC at 10.174.38.89 using userid 'zhmcexporter' defined in HMC credentials file /etc/zhmc-prometheus-exporter/hmccreds.yaml: HTTP authentication failed: No session id was provided (must login) (HMC operation GET /api/services/metrics/context/1, HTTP status 403.4)

There are two issues with that:

• The exporter tries to get metrics without being logged on
• The traceback should not appear in the output, but an error message.

[andy-maier]

I could not reproduce it so far, but I have found a few issues with session management. Some of these issues will be resolved in zhmcclient 1.8.1 (to be released soon) which will become the new minimum version. One other improvement can be made in the exporter.

I'll do some further tests with these fixes and HMC restarts.

[andy-maier]

The dependent package zhmcclient version 1.8.1 has been released.