search

canonical / charm-openstack-service-checks

Collection of Nagios checks and other utilities that can be used to verify the operation of an OpenStack cluster
0 stars 2 forks source link

Some check_http checks missing -S #55

Open sudeephb opened 7 months ago

sudeephb commented 7 months ago

The charm correctly surmised that 80+ checks of openstack services were https and thus required the -S argument but three services, barbican, designate, and gnocchi are failing because the endpoint is https but the check_http command doesn't have -S.

If I check the endpoint list from keystone, I see all of these services have https schema:

$ openstack endpoint list | grep -e barbican -e gnocchi -e designate | 3142eda4d15d47279b89760d5dea984c | XX_PDC2 | barbican | key-manager | True | admin | https://barbican-internal.openstack-pdc-2.xx.com:9312 | | 3a91c5ac34fb4772a09edd16dd978ed5 | XX_PDC2 | designate | dns | True | public | https://designate.openstack-pdc-2.xx.com:9001 | | 3c09135964aa48b8a436a6d65cf63868 | XX_PDC2 | designate | dns | True | internal | https://designate-internal.openstack-pdc-2.xx.com:9001 | | 444cea388ab94ea595e99fc8e4b0b507 | XX_PDC2 | gnocchi | metric | True | internal | https://gnocchi-internal.openstack-pdc-2.xx.com:8041 | | a53cbb734c8042bca108c1ce914352f6 | XX_PDC2 | barbican | key-manager | True | internal | https://barbican-internal.openstack-pdc-2.xx.com:9311 | | b15be86e0b0c4a63919e8b6aa1ccbc08 | XX_PDC2 | designate | dns | True | admin | https://designate-internal.openstack-pdc-2.xx.com:9001 | | c7e1a26378ab4d9fa53689f5613b1d0e | XX_PDC2 | barbican | key-manager | True | public | https://barbican.openstack-pdc-2.xx.com:9311 | | d3fd5b8a8de4422b8e5fd484a07f1d79 | XX_PDC2 | gnocchi | metric | True | admin | https://gnocchi-internal.openstack-pdc-2.xx.com:8041 | | de97d85443214ec6973d2b55509c660a | XX_PDC2 | gnocchi | metric | True | public | https://gnocchi.openstack-pdc-2.xx.com:8041 |

These services were configured with ssl from the beginning but perhaps there is a race condition where a non-https endpoint exists in keystone before it's correctly configured?

I fixed this by going into the debug-hooks environment (it was wanting to run the update-status hook) and clearing the openstack-service-checks.endpoints.configured flag before running the hook:

root@juju-c1611f-28-lxd-4:/var/lib/juju/agents/unit-openstack-service-checks-0/charm# charms.reactive clear_flag openstack-service-checks.endpoints.configured root@juju-c1611f-28-lxd-4:/var/lib/juju/agents/unit-openstack-service-checks-0/charm# hooks/update-status lxc All snaps up to date. /usr/lib/python3/dist-packages/keystoneauth1/adapter.py:179: UserWarning: Using keystoneclient sessions has been deprecated. Please update your software to use keystoneauth1.   warnings.warn('Using keystoneclient sessions has been deprecated. ' root@juju-c1611f-28-lxd-4:/var/lib/juju/agents/unit-openstack-service-checks-0/charm#

Is there a better way to regenerate the service checks from the current keystone endpoints?

Imported from Launchpad using lp2gh.

sudeephb commented 7 months ago

(by zzehring) Are you using the identity-notifications relation between openstack-service-checks and keystone? In theory, that should clear the openstack-service-checks.endpoints.configured flag upon an endpoint update and rerun configure_nrpe_endpoints() function (pretty much exactly what you did in your manual intervention).

sudeephb commented 7 months ago

(by vern) Ah, yes, I am missing that relation.

$ juju status --relations | grep keystone | grep openstack-service-checks keystone:identity-credentials openstack-service-checks:identity-credentials keystone-credentials regular

I was going to say that this must be an fce-templates bug but I see the extra relationship was added back in March: https://git.launchpad.net/fce-templates/commit/config/bundle.yaml?id=cb9b5394de708a01a0ecfa97e3ab23b1e1259930&h=fcb%2Fsku%2Fstable-stein-bionic

I'm not sure why it got dropped from my bundle. Thanks for the pointer.