search

kubernetes-sigs / cluster-api-provider-openstack

Cluster API implementation for OpenStack
https://cluster-api-openstack.sigs.k8s.io/
Apache License 2.0
299 stars 257 forks source link

e2e: `cloud-config` secret can be removed before CAPO/CAPI resources being deleted #2110

Open EmilienM opened 6 months ago

EmilienM commented 6 months ago

/kind bug

What steps did you take and what happened:

In e2e, when tearing down the tests, we remove the namespace that was used. In that namespace, there is the cloud-config secret that is used to store the OpenStack credentials. If that secret is removed, some resources will fail to be reconciled by the cluster so deleting the namespace at the end of the tests is problematic if not everything was reconciled.

This could lead to this error:

E0601 02:15:54.144814       1 controller.go:329] "Reconciler error" err="secrets \"cluster-e2e-ljvg60-cloud-config\" not found" controller="openstackserver" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="OpenStackServer" OpenStackServer="e2e-ljvg60/cluster-e2e-ljvg60-control-plane-plgrj" namespace="e2e-ljvg60" name="cluster-e2e-ljvg60-control-plane-plgrj" reconcileID="588bbb5b-2d05-4391-833c-a2ef4eab3a32"

(Found via https://github.com/kubernetes-sigs/cluster-api-provider-openstack/pull/2067)

I think until now we were lucky because all cluster resources were removed before the namespace gets deleted but with the new controller things might take a little bit more time and we found this issue again.

EmilienM commented 6 months ago

I'm totally unsure if we'll actually fix that situation because CAPI framework waits for a cluster to be deleted as we can see here: https://github.com/kubernetes-sigs/cluster-api/blob/e2bf2b7d108ecd51bf7b23bf50e2432e82d8f2fe/test/framework/cluster_helpers.go#L167-L177

However it seems like nothing waits for the OpenStackServers to be deleted in my initial implementation. I'll play around that for now.

EmilienM commented 6 months ago

/cc @mdbooth

k8s-triage-robot commented 3 months ago

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

EmilienM commented 3 months ago

/remove-lifecycle stale

k8s-triage-robot commented 10 hours ago

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale