CAPD cluster not getting deleted

[cpinjani]

What steps did you take and what happened?

Steps:

• Install turtles & UI extension and import CAPD+RKE2 cluster using fleet & namespace label repo used: https://github.com/rancher-sandbox/rancher-turtles-fleet-example/tree/main
• Cluster is imported into Rancher and becomes Active
• Remove fleet repo and check CAPI cluster deletion status

Status:

Status:
  Conditions:
    Last Transition Time:  2024-08-23T07:44:10Z
    Reason:                Deleting @ Machine/cluster1-control-plane-c97s8
    Severity:              Info
    Status:                False
    Type:                  Ready
    Last Transition Time:  2024-08-23T07:39:48Z
    Status:                True
    Type:                  ControlPlaneInitialized
    Last Transition Time:  2024-08-23T07:44:10Z
    Reason:                Deleting @ Machine/cluster1-control-plane-c97s8
    Severity:              Info
    Status:                False
    Type:                  ControlPlaneReady
    Last Transition Time:  2024-08-23T07:38:55Z
    Status:                True
    Type:                  InfrastructureReady

Logs:

I0823 07:44:05.031628       1 machine.go:539] "Deleting machine container" controller="dockermachine" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="DockerMachine" DockerMachine="default/cluster1-md-0-nfn42-rd7wg" namespace="default" name="cluster1-md-0-nfn42-rd7wg" reconcileID="95eb0ad0-bc02-44c3-bde5-382c8096baf0" Machine="default/cluster1-md-0-nfn42-rd7wg" Machine="default/cluster1-md-0-nfn42-rd7wg" Cluster="default/cluster1"
E0823 07:44:08.515388       1 dockermachine_controller.go:149] "failed to patch DockerMachine" err="failed to patch DockerMachine default/cluster1-md-0-nfn42-rd7wg: dockermachines.infrastructure.cluster.x-k8s.io \"cluster1-md-0-nfn42-rd7wg\" not found" controller="dockermachine" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="DockerMachine" DockerMachine="default/cluster1-md-0-nfn42-rd7wg" namespace="default" name="cluster1-md-0-nfn42-rd7wg" reconcileID="95eb0ad0-bc02-44c3-bde5-382c8096baf0" Machine="default/cluster1-md-0-nfn42-rd7wg" Machine="default/cluster1-md-0-nfn42-rd7wg" Cluster="default/cluster1"
E0823 07:44:08.515462       1 controller.go:329] "Reconciler error" err="failed to patch DockerMachine default/cluster1-md-0-nfn42-rd7wg: dockermachines.infrastructure.cluster.x-k8s.io \"cluster1-md-0-nfn42-rd7wg\" not found" controller="dockermachine" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="DockerMachine" DockerMachine="default/cluster1-md-0-nfn42-rd7wg" namespace="default" name="cluster1-md-0-nfn42-rd7wg" reconcileID="95eb0ad0-bc02-44c3-bde5-382c8096baf0"
2024/08/23 07:44:08 http: TLS handshake error from 10.42.0.1:43104: EOF
2024/08/23 07:44:10 http: TLS handshake error from 10.42.0.1:43106: EOF
I0823 07:44:10.044821       1 machine.go:539] "Deleting machine container" controller="dockermachine" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="DockerMachine" DockerMachine="default/cluster1-control-plane-xsjlr" namespace="default" name="cluster1-control-plane-xsjlr" reconcileID="d6f5a61a-7829-46bc-8add-e639763b0347" Machine="default/cluster1-control-plane-c97s8" Machine="default/cluster1-control-plane-c97s8" Cluster="default/cluster1"
I0823 07:44:10.241025       1 streamwatcher.go:111] Unexpected EOF during watch stream event decoding: unexpected EOF
E0823 07:44:13.552321       1 controller.go:329] "Reconciler error" err="failed to retrieve HAProxy configuration from CustomHAProxyConfigTemplateRef: failed to retrieve custom HAProxy configuration ConfigMap default/cluster1-lb-config: configmaps \"cluster1-lb-config\" not found" controller="dockermachine" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="DockerMachine" DockerMachine="default/cluster1-control-plane-xsjlr" namespace="default" name="cluster1-control-plane-xsjlr" reconcileID="d6f5a61a-7829-46bc-8add-e639763b0347"
E0823 07:44:13.555641       1 controller.go:329] "Reconciler error" err="failed to retrieve HAProxy configuration from CustomHAProxyConfigTemplateRef: failed to retrieve custom HAProxy configuration ConfigMap default/cluster1-lb-config: configmaps \"cluster1-lb-config\" not found" controller="dockermachine" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="DockerMachine" DockerMachine="default/cluster1-control-plane-xsjlr" namespace="default" name="cluster1-control-plane-xsjlr" reconcileID="db3b34ad-2e20-43be-a006-337b187ddeb0"

I0823 07:50:48.127261       1 machine_controller.go:357] "Skipping deletion of Kubernetes Node associated with Machine as it is not allowed" controller="machine" controllerGroup="cluster.x-k8s.io" controllerKind="Machine" Machine="default/cluster1-control-plane-c97s8" namespace="default" name="cluster1-control-plane-c97s8" reconcileID="13584802-471c-4917-ba93-5326e39cfded" RKE2ControlPlane="default/cluster1-control-plane" Cluster="default/cluster1" Node="cluster1-control-plane-c97s8" cause="cluster is being deleted"
I0823 07:50:48.131652       1 machine_controller.go:452] "Waiting for infrastructure to be deleted" controller="machine" controllerGroup="cluster.x-k8s.io" controllerKind="Machine" Machine="default/cluster1-control-plane-c97s8" namespace="default" name="cluster1-control-plane-c97s8" reconcileID="13584802-471c-4917-ba93-5326e39cfded" RKE2ControlPlane="default/cluster1-control-plane" Cluster="default/cluster1" DockerMachine="default/cluster1-control-plane-xsjlr"

What did you expect to happen?

CAPI cluster should get deleted after fleet repo deletion

How to reproduce it?

Steps mentioned above.

Rancher Turtles version

Rancher - v2.9.0 Turtles - v0.9.0 CAPD - v1.7.3

Anything else you would like to add?

On turtles v0.10.0, CAPD version was v1.4.3 so creating cluster using workaround and CAPD v1.3.3, deletion was working fine.

(Please transfer to the respective repo, if not a turtles issue)