oracle / cluster-api-provider-oci

Kubernetes Cluster API Provider for Oracle Cloud Infrastructure
https://oracle.github.io/cluster-api-provider-oci/
Apache License 2.0
40 stars 22 forks source link

Cluster stuck in inconsistent state when nlb is deleted out of lifecycle. #347

Closed sindhusri16 closed 8 months ago

sindhusri16 commented 9 months ago

What happened: User manually deleted the nlb hosting the control plane of the cluster from console. Deletion of this cluster is stuck in an inconsistent state as nlb reconciler is always hitting a 404 not found when searching for this nlb.

What you expected to happen: Cluster should not be stuck in 'deleting' phase, it could reconcile the nlb as deleted and go forward with deletion.

How to reproduce it (as minimally and precisely as possible): Create a cluster and delete the nlb-apiserver from console, then delete the cluster.

Anything else we need to know?: Logs: kubectl get events -A | grep oke-amaaaaaakdrwrhiamr4gonrrnvrts6bs oke 14m Normal FailureDomainsReady ocicluster/oke-amaaaaaakdrwrhiamr4gonrrnvrts6bs FailureDomain is in ready state oke 48m Warning ReconcileError ocicluster/oke-amaaaaaakdrwrhiamr4gonrrnvrts6bs failed to reconcile Api Server Network Loadbalancer: Error returned by NetworkLoadBalancer Service. Http Status Code: 404. Error Code: NotAuthorizedOrNotFound. Opc request id: . Message: Authorization failed or requested resource not found.... oke 31m Warning ReconcileError ocicluster/oke-amaaaaaakdrwrhiamr4gonrrnvrts6bs failed to reconcile Api Server Network Loadbalancer: Error returned by NetworkLoadBalancer Service. Http Status Code: 404. Error Code: NotAuthorizedOrNotFound. Opc request id: . Message: Authorization failed or requested resource not found.... oke 14m Warning ReconcileError ocicluster/oke-amaaaaaakdrwrhiamr4gonrrnvrts6bs failed to reconcile Api Server Network Loadbalancer: Error returned by NetworkLoadBalancer Service. Http Status Code: 404. Error Code: NotAuthorizedOrNotFound. Opc request id: . Message: Authorization failed or requested resource not found.

Environment:

sindhusri16 commented 9 months ago

Could the team please confirm the ETA for this fix.

shyamradhakrishnan commented 9 months ago

Approx ETA: 28/Feb/2024