check_cinder_services.py broken in Xena and Yoga

[sudeephb]

/usr/local/lib/nagios/plugins/check_cinder_services.py <function check_cinder_services at 0x7f6561a84310> raised unknown exception '<class 'keystoneauth1.exceptions.catalog.EndpointNotFound'>'

Traceback (most recent call last): File "/usr/local/lib/nagios/plugins/nagios_plugin3.py", line 37, in try_check function(*args, kwargs) File "/usr/local/lib/nagios/plugins/check_cinder_services.py", line 35, in check_cinder_services services = cinder.get("/os-services").json()["services"] File "/usr/lib/python3/dist-packages/keystoneauth1/adapter.py", line 386, in get return self.request(url, 'GET', kwargs) File "/usr/lib/python3/dist-packages/openstack/proxy.py", line 94, in request response = super(Proxy, self).request( File "/usr/lib/python3/dist-packages/keystoneauth1/adapter.py", line 248, in request return self.session.request(url, method, **kwargs) File "/usr/lib/python3/dist-packages/keystoneauth1/session.py", line 797, in request raise exceptions.EndpointNotFound() keystoneauth1.exceptions.catalog.EndpointNotFound: Could not find requested endpoint in Service Catalog.

---

Imported from Launchpad using lp2gh.

• date created: 2022-03-03T20:19:48Z

• owner: gustavosr98

• assignee: jneo8

• the launchpad url

[sudeephb]

(by gustavosr98) Could be related to v2 cinder endpoint deprecated and using v3

https://opendev.org/openstack/charm-cinder/commit/bc48c60f1cd0bf7ceb8813dced504f5e0d37d829 https://bugs.launchpad.net/charm-cinder/+bug/1947387

[sudeephb]

(by phausman) I have hit this bug on Yoga too. It looks related to API version.

The issue is causing Nagios to report "Unknown" status on juju-openstack-service-checks-0 host, with the message "<function check_cinder_services at 0x7f1cd41b0160> raised unknown exception '<class 'keystoneauth1.exceptions.catalog.EndpointNotFound'>'".

As a test, I have added export OS_VOLUME_API_VERSION=3 to /var/lib/nagios/nagios.novarc and it temporarily fixes the problem.

Steps to reproduce:

$ source /var/lib/nagios/nagios.novarc

$ openstack endpoint list \
  -c "Service Name" \
  -c "Service Type" \
  -c Interface \
  -c URL \
  -f value | grep cinder
cinderv3 volumev3 internal https://cinder-internal.redacted.com:8776/v3/$(tenant_id)s
cinderv3 volumev3 public https://cinder.redacted.com:8776/v3/$(tenant_id)s
cinderv3 volumev3 admin https://cinder.redacted.com:8776/v3/$(tenant_id)s

$ /usr/local/lib/nagios/plugins/check_cinder_services.py
<function check_cinder_services at 0x7f324d8c73a0> raised unknown exception '<class 'keystoneauth1.exceptions.catalog.EndpointNotFound'>'
============================================================
Traceback (most recent call last):
  File "/usr/local/lib/nagios/plugins/nagios_plugin3.py", line 37, in try_check
    function(*args, **kwargs)
  File "/usr/local/lib/nagios/plugins/check_cinder_services.py", line 35, in check_cinder_services
    services = cinder.get("/os-services").json()["services"]
  File "/usr/lib/python3/dist-packages/keystoneauth1/adapter.py", line 386, in get
    return self.request(url, 'GET', **kwargs)
  File "/usr/lib/python3/dist-packages/openstack/proxy.py", line 94, in request
    response = super(Proxy, self).request(
  File "/usr/lib/python3/dist-packages/keystoneauth1/adapter.py", line 248, in request
    return self.session.request(url, method, **kwargs)
  File "/usr/lib/python3/dist-packages/keystoneauth1/session.py", line 797, in request
    raise exceptions.EndpointNotFound()
keystoneauth1.exceptions.catalog.EndpointNotFound: Could not find requested endpoint in Service Catalog.
============================================================

$ export OS_VOLUME_API_VERSION=3

$ /usr/local/lib/nagios/plugins/check_cinder_services.py
OK: All cinder services happy

[sudeephb]

(by jneo8) Cinder document: https://docs.openstack.org/python-cinderclient/latest/user/shell.html