Add `start` and `stop` command to `kind`.

[joshrosso]

What would you like to be added:

Add a start and stop command to kind.

This would be a light abstraction on docker kill or docker stop and docker start. It would also do some validation that start/stop is supported for the cluster (e.g. is it multi-node? etc).

note, I've not considered podman support, but imagine this could be possible

Why is this needed:

For the sake of local resources, I sometimes use docker kill to turn down running (single-node) kind clusters and docker start to bring them back up. It could be nice to do this entirely through kind.

Curious if the kind project would be interested in a contribution around this functionality! No worries if not, thanks for your consideration :smile:!

[BenTheElder]

For the sake of local resources, I sometimes use docker kill to turn down running (single-node) kind clusters and docker start to bring them back up. It could be nice to do this entirely through kind.

Are you sure this is working properly? What host environment?

IIRC in the past we did not do this because it didn't actually work correctly. That may not be true today due to various fixes ...

[aojea]

IIRC it works for single node, but it will fail for multi-node if the runtime assign different IPs to the node

[tao12345666333]

We did have some attempts before, but unfortunately not all problems were solved in the end, more details can refer to #408 and #484

[BenTheElder]

seems like this is on the backlog until #1689 is fixed

[jaredallard]

In case it's interesting we solved this in our fork of KinD for single node clusters by letting Docker assign a stable IP address and then using it to prevent issues with IP address changes. (https://github.com/getoutreach/kind/pull/4 and https://github.com/getoutreach/kind/pull/5)

It's not perfect but it works.

One of these days I want to pitch the few changes we made in PRs here, but just haven't gotten to It so hopefully some of that code is useful for anyone wanting to implement a start/stop until it's done here.

[BenTheElder]

Thanks for sharing!

One of these days I want to pitch the few changes we made in PRs here, but just haven't gotten to It so hopefully some of that code is useful for anyone wanting to implement a start/stop until it's done here.

If you do, please see our contributing guide https://kind.sigs.k8s.io/docs/contributing/getting-started/ In particular we'd prefer to discuss approach in detail before moving to feature PRs.

Also FWIW there's promising alternatives in #2671 (though, that reallly should have been discussed more up front as well, and probably same thing for the changes being PRed to kubeadm).

[cjc7373]

Since multi-node cluster (with a single control plane node) is working after restarting docker. I think it is enough to just call docker stop/start to implement this?

[BenTheElder]

I think it is enough to just call docker stop/start to implement this?

You can ~kind get nodes | xargs docker stop ... for now but we also have podman where this won't work for multi-node clusters IIRC, nerdctl where I'm not sure, and in all cases where HA doesn't work

2272 #1689 amongst others

[jaredallard]

Note if you start and stop a kind node in Docker, the IP will change for the container, which breaks the cluster. At least that was the case when I last worked in an environment using KinD in 2023, so that's what required the changes I originally ~made~ shared.

Sorry I never got around to starting a discussion around those changes before I stopped working there 😿

[BenTheElder]

Note if you start and stop a kind node in Docker, the IP will change for the container, which breaks the cluster. At least that was the case when I last worked in an environment using KinD in 2023

That shouldn't have been breaking kind even in 2023. It was mitigated in may 2022. https://github.com/kubernetes-sigs/kind/pull/2775

There are other issues though, with podman (and maybe nerdctl) and with multiple-control-plane nodes / "HA" (see above).