canonical / mysql-k8s-operator

A Charmed Operator for running MySQL on Kubernetes
https://charmhub.io/mysql-k8s
Apache License 2.0
8 stars 15 forks source link

[DPE-4118] Address drained units rejoining the cluster with a new PV #433

Open shayancanonical opened 3 weeks ago

shayancanonical commented 3 weeks ago

Issue

https://github.com/canonical/mysql-k8s-operator/issues/407 When a node is drained along with the pods on that node + the PV of the unit is deleted, then when the unit is restarted, the unit tries to rejoin the cluster. However, there is already metadata for that unit in the cluster metadata that prevents the unit's SST.

Solution

If the instance is already in non-online state when trying to join the cluster, run cluster.rescan() on the primary node so that the rejoining unit's SST completes without errors.

shayancanonical commented 1 day ago

Added an integration test in ecd1150