flyersa
commented
3 weeks ago as example purging all nova_libvirt containers and volumes including the secrets volume on a compute node, on a new rollout you see this debug output:
TASK [nova-cell : Pushing nova secret xml for libvirt] *************************
ok: [open-cmpt01] => (item={'uuid': '2989ffe6-a3aa-48df-b0d8-b5a472e13904', 'name': 'client.nova secret', 'enabled': True})
ok: [open-cmpt01] => (item={'uuid': '5a6876ce-21a4-11ef-ad42-772fa17dbf00', 'name': 'client.cinder secret', 'enabled': 'yes'})
the client.cinder secret here as example is 100% correct in the ansible output as it is configured. But actually after deployment on the container you see this:
(nova-libvirt)[root@open-cmpt01 /]# virsh secret-list
UUID Usage
-------------------------------------------------------------------
00e0db8c-a3b3-4667-9f6f-2b5edf4400b7 ceph client.cinder secret
18a088ab-7180-4407-b112-3ec93cadc517 ceph client.nova secretfun fact the files are also created:
(nova-libvirt)[root@open-cmpt01 secrets]# ls -l /etc/libvirt/secrets/
total 32
-rw------- 1 root root 41 Jun 7 16:28 00e0db8c-a3b3-4667-9f6f-2b5edf4400b7.base64
-rw------- 1 root root 170 Jun 7 16:28 00e0db8c-a3b3-4667-9f6f-2b5edf4400b7.xml
-rw------- 1 root root 41 Jun 7 16:28 18a088ab-7180-4407-b112-3ec93cadc517.base64
-rw------- 1 root root 168 Jun 7 16:28 18a088ab-7180-4407-b112-3ec93cadc517.xml
-rw------- 1 root root 41 Jun 7 16:28 2989ffe6-a3aa-48df-b0d8-b5a472e13904.base64
-rw------- 1 root root 168 Jun 7 16:28 2989ffe6-a3aa-48df-b0d8-b5a472e13904.xml
-rw------- 1 root root 41 Jun 7 16:28 5a6876ce-21a4-11ef-ad42-772fa17dbf00.base64
-rw------- 1 root root 170 Jun 7 16:28 5a6876ce-21a4-11ef-ad42-772fa17dbf00.xml
but they are not used. something is wrong here
changing them manuel works then but as i said that is not update/rollout proof:
virsh secret-undefine 00e0db8c-a3b3-4667-9f6f-2b5edf4400b7
virsh secret-define --file /etc/libvirt/secrets/5a6876ce-21a4-11ef-ad42-772fa17dbf00.xml
virsh secret-set-value --secret 5a6876ce-21a4-11ef-ad42-772fa17dbf00 --base64 AQDYyGFmLUh0AhAA+MQ2C9LY/8GjvHLFa7CP7g==
OSISM release version
latest
What's the problem?
In our past last 3-5 deployments we always have the issue that with external ceph that the virsh secrets in the nova containers for the cinder client do not match the ceph fsid despite being configured as cinder_rbd_secret_uuid in configuration.yml and in the secrets file.
This leads to issues deploying the system with external ceph which is not managed by osism. We could not figure out where the arbitary uuids come from (they are always the same).
The only workaround so far is to change the virsh secrets manual in the containers which is not update or rollout proof.
References to existing reports
References to existing bug reports, mailing lists, ...
Severity
low
Urgency
low