search

kubernetes / minikube

Run Kubernetes locally
https://minikube.sigs.k8s.io/
Apache License 2.0
29.22k stars 4.87k forks source link

unable to start minikube with docker without cri-dockerd #14410

Closed sudlo closed 2 years ago

sudlo commented 2 years ago

What Happened?

I have installed minikube on aws ec2 instance followed all the steps. Please find the logs attached for your reference

Attach the log file

root@ip-172-31-47-92:~# cat logs.txt *

stderr: I0624 04:02:49.147151 6897 exec_runner.go:51] Run: sudo systemctl is-active --quiet service kubelet I0624 04:02:49.156870 6897 fix.go:103] recreateIfNeeded on minikube: state=Stopped err= W0624 04:02:49.156885 6897 fix.go:129] unexpected machine state, will restart: I0624 04:02:49.159426 6897 out.go:177] 🔄 Restarting existing none bare metal machine for "minikube" ... I0624 04:02:49.162619 6897 profile.go:148] Saving config to /root/.minikube/profiles/minikube/config.json ... I0624 04:02:49.162753 6897 start.go:306] post-start starting for "minikube" (driver="none") I0624 04:02:49.162762 6897 start.go:316] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs] I0624 04:02:49.162803 6897 exec_runner.go:51] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs I0624 04:02:49.169850 6897 main.go:134] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found I0624 04:02:49.169872 6897 main.go:134] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found I0624 04:02:49.169883 6897 main.go:134] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found I0624 04:02:49.175670 6897 out.go:177] ℹ️ OS release is Ubuntu 18.04.6 LTS I0624 04:02:49.180296 6897 filesync.go:126] Scanning /root/.minikube/addons for local assets ... I0624 04:02:49.180353 6897 filesync.go:126] Scanning /root/.minikube/files for local assets ... I0624 04:02:49.180373 6897 start.go:309] post-start completed in 17.610894ms I0624 04:02:49.180382 6897 fix.go:57] fixHost completed within 45.605194ms I0624 04:02:49.180388 6897 start.go:81] releasing machines lock for "minikube", held for 45.62773ms I0624 04:02:49.180936 6897 exec_runner.go:51] Run: /bin/bash -c "sudo mkdir -p /etc && printf %!s(MISSING) "runtime-endpoint: unix:///var/run/cri-dockerd.sock image-endpoint: unix:///var/run/cri-dockerd.sock " | sudo tee /etc/crictl.yaml" I0624 04:02:49.181000 6897 exec_runner.go:51] Run: curl -sS -m 2 https://k8s.gcr.io/ I0624 04:02:49.202779 6897 exec_runner.go:51] Run: sudo systemctl unmask docker.service I0624 04:02:49.367152 6897 exec_runner.go:51] Run: sudo systemctl enable docker.socket I0624 04:02:49.510093 6897 exec_runner.go:51] Run: sudo systemctl daemon-reload I0624 04:02:49.686716 6897 exec_runner.go:51] Run: sudo systemctl restart docker I0624 04:02:49.964699 6897 exec_runner.go:51] Run: sudo systemctl enable cri-docker.socket I0624 04:02:49.977302 6897 out.go:177] W0624 04:02:49.980229 6897 out.go:239] ❌ Exiting due to RUNTIME_ENABLE: sudo systemctl enable cri-docker.socket: exit status 1 stdout:

stderr: Failed to enable unit: Unit file cri-docker.socket does not exist.

W0624 04:02:49.980260 6897 out.go:239] W0624 04:02:49.984043 6897 out.go:239] ╭───────────────────────────────────────────────────────────────────────────────────────────╮ │ │ │ 😿 If the above advice does not help, please let us know: │ │ 👉 https://github.com/kubernetes/minikube/issues/new/choose │ │ │ │ Please run minikube logs --file=logs.txt and attach logs.txt to the GitHub issue. │ │ │ ╰───────────────────────────────────────────────────────────────────────────────────────────╯ I0624 04:02:49.986533 6897 out.go:177]

Operating System

Other

Driver

No response

spowelljr commented 2 years ago

Hi @sudlo, thanks for reporting your issue with minikube.

Kubernetes v1.24 dropped support for Dockershim, so if you want to use the combination of the none driver, Kubernetes v1.24+, and the Docker container runtime you'll need to install cri-dockerd on your system, as listed in our requirements page.

Other options include using a different driver, using an older version of Kubernetes --kubernetes-version, or using a different container runtime --container-runtime.

Please try one of those solution and let me know if it resolves your issue, thanks

sudlo commented 2 years ago

thanks a lot let me try it again

afbjorklund commented 2 years ago

Some more details in #14403, the docker driver should check for both cri-dockerd and crictl.

Currently it just looks for the "docker" client (?), but not for "dockerd" and "cri-dockerd" servers

// Available returns an error if it is not possible to use this runtime on a host
func (r *Docker) Available() error {
        _, err := exec.LookPath("docker")
        return err
}
spowelljr commented 2 years ago

Some more details in #14403, the docker driver should check for both cri-dockerd and crictl.

Currently it just looks for the "docker" client (?), but not for "dockerd" and "cri-dockerd" servers

// Available returns an error if it is not possible to use this runtime on a host
func (r *Docker) Available() error {
        _, err := exec.LookPath("docker")
        return err
}

That's a good point, I created an issue for this request https://github.com/kubernetes/minikube/issues/14460

spowelljr commented 2 years ago

So there are two error cases:

none driver

Users are unaware that minikube v1.26.0 requires a new dependency of cri-docker.

Background:

Kubernetes v1.24 dropped support for Dockershim, so if you want to use the combination of the none driver, Kubernetes v1.24+, and the Docker container runtime you'll need to install cri-dockerd on your system, as listed in our requirements page.

Other options include using a different driver, using an older version of Kubernetes --kubernetes-version, or using a different container runtime --container-runtime.

Implementing https://github.com/kubernetes/minikube/issues/14460 will detect that they're missing a system dependency. Will this prevent us from needing a solution message?

Solution: Installing cri-docker, installation instructions: https://github.com/Mirantis/cri-dockerd#build-and-install

Other drivers (docker, etc.)

Users that try doing minikube start --kubernetes-version=stable (or v1.24.0+) on a cluster previously created before minikube v1.26.0 can result in failing to find cri-docker. I've tried reproducing it but oddly wasn't able to do so, but we have logs of users experiencing it (https://github.com/kubernetes/minikube/issues/14411).

Solution: Run minikube delete then try starting minikube again and all should be resolved

robrich commented 2 years ago

Please link to this as a known issue in the 1.26.0 release page. Something like this:

Using a minikube cluster created before 1.26.0? After upgrading, you'll likely need to delete & recreate it. See issue 14410

spowelljr commented 2 years ago

Please link to this as a known issue in the 1.26.0 release page.

Updated, thanks for the idea

shubham-cmyk commented 2 years ago

You need to install the cri-dockerd, We will build the cri-dockerd from scratch.

Clone the repo: git clone https://github.com/Mirantis/cri-dockerd.git

Install Golang Skip If present :

  1. wget https://storage.googleapis.com/golang/getgo/installer_linux
  2. chmod +x ./installer_linux
  3. ./installer_linux
  4. source ~/.bash_profile

Build the cri-dockerd :

  1. cd cri-dockerd
  2. mkdir bin
  3. go get && go build -o bin/cri-dockerd
  4. mkdir -p /usr/local/bin
  5. install -o root -g root -m 0755 bin/cri-dockerd /usr/local/bin/cri-dockerd
  6. cp -a packaging/systemd/* /etc/systemd/system
  7. sed -i -e 's,/usr/bin/cri-dockerd,/usr/local/bin/cri-dockerd,' /etc/systemd/system/cri-docker.service
  8. systemctl daemon-reload
  9. systemctl enable cri-docker.service
  10. systemctl enable --now cri-docker.socket`
Technicalnikhi commented 1 year ago

minikube type: Control Plane host: Stopped kubelet: Stopped apiserver: Stopped kubeconfig: Stopped getting error while checking status of minikube

spowelljr commented 1 year ago

Hi @Technicalnikhi, could you open a new issue and follow the template in the issue to upload all required logs, thanks

shu-mutou commented 1 year ago

cri-docker.service will always fails at first time.

Because sudo systemctl start cri-docker.service seems to create /etc/systemd/system/cri-docker.service.d/10-cni.conf, but the created file calls /usr/bin/cri-dockerd.

Despite the installation guide rewrite tha path to cri-dockerd. :crying_cat_face:

So we need to modify the path to the cri-dockerd in /etc/systemd/system/cri-docker.service.d/10-cni.conf after the first execution. Then run sudo systemctl daemon-reload && sudo systemctl restart cri-docker

shu-mutou commented 1 year ago

I don't know what timing, but 10-cni.conf seems to be restored. :disappointed: So in the installation guide, install ... should be set to /usr/bin/cri-dockerd and sed should not be executed.

afbjorklund commented 1 year ago
shu-mutou commented 1 year ago

@afbjorklund Thank you so much for your explanation! I understand the current situation.

owenallenaz commented 1 year ago

Is it possible to get some clarification on whether cri-dockerd is required with using the --driver=docker. The docs don't mention it. If it truly is required it would be great if the official docs are updated. I'm receiving errors stating it's required using minikube 1.30.1 and kubectl 1.26.5.

afbjorklund commented 1 year ago

@owenallenaz : it is not required to install cri-dockerd on the host running the docker driver... it is required in the image that runs kubernetes, but that one (kicbase) is provided by minikube