Fix intermittent connection refused

[fmount]

During the test of Glance image sign that rely on Barbican to store secrets, we’d see Connection refused errors on about 60% of the calls made to the service. This let the image signing fail most of the times. In addition, simply running curl against the barbican-internal endpoint shows this issue, because it fails more than half of the times with a connection refused error on 9311. However, if we point directly to the Pod (bypassing the k8s Service), we can see that this problem doesn't occur, and we're able to always reach the Barbican service and successfully sign the Glance images. The root cause of this issue is the k8s Service sending traffic to the BarbicanAPI, as well as keystoneListener and BarbicanWorker: these two services are not listening on 9311, and this result in a connection refused on that port. The BarbicanAPI service should be built with a selector that is able to forward the traffic to barbicanAPI only, and this patch improves the way labels are used to achieve this goal, similarly to what we did in Cinder, Glance and Manila. With this patch we no longer see any issue when we try to reach the barbicanAPI component and we're able to retrieve the secrets and sign images.

Fixes: OSPRH-3420

[fmount]

With the change of this patch we can see:

barbican-api-api-6b47bbc6cc-5qplw                              2/2     Running     0          73m    component=barbican-api,pod-template-hash=6b47bbc6cc,service=barbican
barbican-keystone-listener-keystone-listener-dddb5786f-7qjld   2/2     Running     0          73m    component=keystone-listener,pod-template-hash=dddb5786f,service=barbican
barbican-worker-worker-c7d8c877-hprt6                          2/2     Running     0          73m    component=barbican-worker,pod-template-hash=c7d8c877,service=barbican

and the service has the right selector:

$ oc get svc barbican-internal -o jsonpath='{ .spec.selector }'
{"component":"barbican-api","service":"barbican"}

and the traffic is redirected to the right component:

$ while true; do curl barbican-internal.openstack.svc:9311; printf "\n"; sleep 4; done
{"versions": {"values": [{"id": "v1", "status": "stable", "links": [{"rel": "self", "href": "https://barbican.openstack.svc:9311/v1/"}, {"rel": "describedby", "type": "text/html", "href": "https://docs.openstack.org/"}], "media-types": [{"base": "application/json", "type": "applic
ation/vnd.openstack.key-manager-v1+json"}]}]}}
{"versions": {"values": [{"id": "v1", "status": "stable", "links": [{"rel": "self", "href": "https://barbican.openstack.svc:9311/v1/"}, {"rel": "describedby", "type": "text/html", "href": "https://docs.openstack.org/"}], "media-types": [{"base": "application/json", "type": "applic
ation/vnd.openstack.key-manager-v1+json"}]}]}}
{"versions": {"values": [{"id": "v1", "status": "stable", "links": [{"rel": "self", "href": "https://barbican.openstack.svc:9311/v1/"}, {"rel": "describedby", "type": "text/html", "href": "https://docs.openstack.org/"}], "media-types": [{"base": "application/json", "type": "applic
ation/vnd.openstack.key-manager-v1+json"}]}]}}
{"versions": {"values": [{"id": "v1", "status": "stable", "links": [{"rel": "self", "href": "https://barbican.openstack.svc:9311/v1/"}, {"rel": "describedby", "type": "text/html", "href": "https://docs.openstack.org/"}], "media-types": [{"base": "application/json", "type": "applic
ation/vnd.openstack.key-manager-v1+json"}]}]}}
{"versions": {"values": [{"id": "v1", "status": "stable", "links": [{"rel": "self", "href": "https://barbican.openstack.svc:9311/v1/"}, {"rel": "describedby", "type": "text/html", "href": "https://docs.openstack.org/"}], "media-types": [{"base": "application/json", "type": "applic
ation/vnd.openstack.key-manager-v1+json"}]}]}}
{"versions": {"values": [{"id": "v1", "status": "stable", "links": [{"rel": "self", "href": "https://barbican.openstack.svc:9311/v1/"}, {"rel": "describedby", "type": "text/html", "href": "https://docs.openstack.org/"}], "media-types": [{"base": "application/json", "type": "applic
ation/vnd.openstack.key-manager-v1+json"}]}]}}
{"versions": {"values": [{"id": "v1", "status": "stable", "links": [{"rel": "self", "href": "https://barbican.openstack.svc:9311/v1/"}, {"rel": "describedby", "type": "text/html", "href": "https://docs.openstack.org/"}], "media-types": [{"base": "application/json", "type": "applic
ation/vnd.openstack.key-manager-v1+json"}]}]}}
{"versions": {"values": [{"id": "v1", "status": "stable", "links": [{"rel": "self", "href": "https://barbican.openstack.svc:9311/v1/"}, {"rel": "describedby", "type": "text/html", "href": "https://docs.openstack.org/"}], "media-types": [{"base": "application/json", "type": "applic
ation/vnd.openstack.key-manager-v1+json"}]}]}}
{"versions": {"values": [{"id": "v1", "status": "stable", "links": [{"rel": "self", "href": "https://barbican.openstack.svc:9311/v1/"}, {"rel": "describedby", "type": "text/html", "href": "https://docs.openstack.org/"}], "media-types": [{"base": "application/json", "type": "applic
ation/vnd.openstack.key-manager-v1+json"}]}]}}
{"versions": {"values": [{"id": "v1", "status": "stable", "links": [{"rel": "self", "href": "https://barbican.openstack.svc:9311/v1/"}, {"rel": "describedby", "type": "text/html", "href": "https://docs.openstack.org/"}], "media-types": [{"base": "application/json", "type": "applic
ation/vnd.openstack.key-manager-v1+json"}]}]}}
...
...

[openshift-ci[bot]]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: abays, fmount, konan-abhi

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files: - ~~[OWNERS](https://github.com/openstack-k8s-operators/barbican-operator/blob/main/OWNERS)~~ [abays] Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment