search

kubernetes-sigs / kubespray

Deploy a Production Ready Kubernetes Cluster
Apache License 2.0
15.86k stars 6.41k forks source link

Avoid Kubespray to delete docker #10530

Closed hajmehdikabir closed 3 months ago

hajmehdikabir commented 10 months ago

What would you like to be added: Hello. Is there any way to avoid delete docker during upgrade from docker to containerd?

I think we can add a variable to disable it.

Why is this needed:

When we want to upgrade kubernetes from 1.23.5 to newer version (I know docker is not supported any more and we should use containerd), in upgrade procedure kubespray will delete docker.

https://github.com/kubernetes-sigs/kubespray/blob/master/roles/container-engine/validate-container-engine/tasks/main.yml#L103

Unfortunately we have containers (like ceph containers) that still use docker. And when kubespray deletes docker, those containers are stopped and CSI doesn't work any more.

disco-stu commented 10 months ago

I've took a look at the corresponding code and would like to work on this.

As far as I can see, the code has multiple "uninstall" steps for various runtimes as well as runc and skopeo. My approach would be to add a new variable (list) that can be used to skip certain uninstall steps and isn't limited to docker.

Please let me know what do you think about this approach.

hajmehdikabir commented 10 months ago

I think this is good idea. Thank you!

vipcxj commented 10 months ago

The same issue, Is installing the docker manaully again a workaround? Will it break the k8s?

hajmehdikabir commented 10 months ago

@vipcxj I don't think so. I'll try it.

hajmehdikabir commented 10 months ago

@vipcxj I Commented Uninstall docker task and ran ansible. Unfortunately docker service failed. because it has conflicts with new containers (those containerd runned them)

But cluster is healthy and OK. Then I changed container_engine to docker and re-deployed it to bring ceph containers back.

win5923 commented 9 months ago

I restore Docker using the following command. it not break the k8s.

sudo apt remove docker && sudo apt install docker docker-ce
sudo service docker start
hajmehdikabir commented 9 months ago

@win5923 I think it's not safe

k8s-triage-robot commented 6 months ago

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot commented 5 months ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

cheetahfox commented 4 months ago

TLDR; Running Debian based system and want to run Ceph on nodes also running k8s installed with Kubespray; use Podman and not Docker.

I have been struggling with a similar issue in my home lab and I think there are few key context points missing from the discussion so far.

Let me start first with saying (as other people have reported) you can make both docker and containerd containers co-exist. It works and honestly with very little in the way of problems while actually running, but there is a big issue...

What doesn't doesn't work (most of the time) is the suggestion above keeping Docker installed by preventing it from being uninstalled in the validate-container-engine task. I implemented this suggestion and was a little surprised to find that it was actually getting removed in a totally different task.

The problem here isn't keeping Kubespray from removing docker, the problem is that it's an OS packaged version of docket that most people will be installing.

On Debian based systems (only things I have really looked at), if you install docker.io or docker-ce it's going to pull in the required dependencies which include runc and containderd. This right here is the problem. Because Kubespray is installing these two items directly and not using any OS packages, it checks to see if they are installed as an OS package. And correctly removes to OS packages if it finds them installed. Which also removes docker!

Since Kubespray is installing a specific version of these dependencies you can not install a OS package that includes docker since it's going to overwrite those; it does seem to work ok in most cases but breaks everything with Kubespray pinning those versions.

In general this means that unless someone is installing docker not using OS packages the idea of keeping Kubespray from removing docker isn't ever going to be effective.

If you are like me and wanted to keep docker around for Ceph and you are running a Debian based OS; I do have a solution.

Use Podman and not docker. Podman uses crun as it's dependency so Kubespray doesn't ever do anything that would remove Podman directly or indirectly. So you can still use ceph and it's containers and Kubespray won't mess with them. Granted; I haven't had much time to run this so maybe there is an interaction I don't know about yet but I expect it will work for you.

k8s-triage-robot commented 3 months ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

k8s-ci-robot commented 3 months ago

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to [this](https://github.com/kubernetes-sigs/kubespray/issues/10530#issuecomment-2133825883): >The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. > >This bot triages issues according to the following rules: >- After 90d of inactivity, `lifecycle/stale` is applied >- After 30d of inactivity since `lifecycle/stale` was applied, `lifecycle/rotten` is applied >- After 30d of inactivity since `lifecycle/rotten` was applied, the issue is closed > >You can: >- Reopen this issue with `/reopen` >- Mark this issue as fresh with `/remove-lifecycle rotten` >- Offer to help out with [Issue Triage][1] > >Please send feedback to sig-contributor-experience at [kubernetes/community](https://github.com/kubernetes/community). > >/close not-planned > >[1]: https://www.kubernetes.dev/docs/guide/issue-triage/ Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.