clusterctl alpha rollout restart kthreescontrolplane support

[lukebond]

For kubeadm clusters one can do this:

$ clusterctl alpha rollout restart kubeadmcontrolplane/my-cluster

(see https://cluster-api.sigs.k8s.io/clusterctl/commands/alpha-rollout)

...and it will replace the controlplane like it would a machine deployment. it would be nice if we could support this! i'll go read up on it and dump some things in here to gather requirements and spec it out.

[lukebond]

was looking at the kubeadm implementation; sharing a few observations:

• clusterctl alpha rollout restart kubeadmcontrolplane/... is something that needs to be implemented in the clusterctl CLI as well as server-side
• the client just needs to set the RolloutAfter field in the spec, and check it wasn't paused at the time
• the server makes a note in the status of the CP to say that a rollout is needed. that check is either because the config is out of date or if the RolloutAfter field is set
• that code seems to exist in the k3s provider control plane controller, so i suspect this would "Just work" if we could just call it from clusterctl. i will try setting RolloutAfter on a cluster and see what happens
• that same code is supposed to roll CPs when the config has drifted and i don't see that happening in our clusters so i suspect something is wrong. if i find that to be true i'll open an issue

[lukebond]

i edited a KCP, setting spec.upgradeAfter and it did a rollout. brilliant! guess we just need to implement it in the clusterctl CLI.

[lukebond]

this explains why a configuration change doesn't trigger a rollout!

[lukebond]

Created an issue for it: https://github.com/cluster-api-provider-k3s/cluster-api-k3s/issues/83