search

kubernetes / minikube

Run Kubernetes locally
https://minikube.sigs.k8s.io/
Apache License 2.0
29.19k stars 4.87k forks source link

Unable to start minikube (X Exiting due to K8S_INSTALL_FAILED) #13549

Closed kumaranraj2895 closed 2 years ago

kumaranraj2895 commented 2 years ago

What Happened?

X Exiting due to K8S_INSTALL_FAILED: updating control plane: downloading binaries: downloading kubeadm: download failed: https://storage.googleapis.com/kubernetes-release/release/v1.23.1/bin/linux/amd64/kubeadm?checksum=file:https://storage.googleapis.com/kubernetes-release/release/v1.23.1/bin/linux/amd64/kubeadm.sha256: getter: &{Ctx:context.Background Src:https://storage.googleapis.com/kubernetes-release/release/v1.23.1/bin/linux/amd64/kubeadm?checksum=file:https://storage.googleapis.com/kubernetes-release/release/v1.23.1/bin/linux/amd64/kubeadm.sha256 Dst:/home/developer/.minikube/cache/linux/v1.23.1/kubeadm.download Pwd: Mode:2 Umask:---------- Detectors:[0x358be48 0x358be48 0x358be48 0x358be48 0x358be48 0x358be48 0x358be48] Decompressors:map[bz2:0x358be48 gz:0x358be48 tar:0x358be48 tar.bz2:0x358be48 tar.gz:0x358be48 tar.xz:0x358be48 tar.zst:0x358be48 tbz2:0x358be48 tgz:0x358be48 txz:0x358be48 tzst:0x358be48 xz:0x358be48 zip:0x358be48 zst:0x358be48] Getters:map[file:0xc0005b1e30 http:0xc000580960 https:0xc0005809c0] Dir:false ProgressListener:0x354c130 Insecure:false Options:[0x18a59e0]}: invalid checksum: Error downloading checksum file: Get "https://storage.googleapis.com/kubernetes-release/release/v1.23.1/bin/linux/amd64/kubeadm.sha256": x509: certificate signed by unknown authority

Attach the log file

I0203 11:52:55.431700 31270 cli_runner.go:133] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube I0203 11:52:55.468395 31270 main.go:130] libmachine: Using SSH client type: native I0203 11:52:55.468580 31270 main.go:130] libmachine: &{{{ 0 [] [] []} docker [0x7a0a80] 0x7a3b60 [] 0s} 127.0.0.1 49157 } I0203 11:52:55.468597 31270 main.go:130] libmachine: About to run SSH command:

            if ! grep -xq '.*\sminikube' /etc/hosts; then
                    if grep -xq '127.0.1.1\s.*' /etc/hosts; then
                            sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 minikube/g' /etc/hosts;
                    else
                            echo '127.0.1.1 minikube' | sudo tee -a /etc/hosts;
                    fi
            fi

I0203 11:52:55.610195 31270 main.go:130] libmachine: SSH cmd err, output: : I0203 11:52:55.610236 31270 ubuntu.go:175] set auth options {CertDir:/home/developer/.minikube CaCertPath:/home/developer/.minikube/certs/ca.pem CaPrivateKeyPath:/home/developer/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/developer/.minikube/machines/server.pem ServerKeyPath:/home/developer/.minikube/machines/server-key.pem ClientKeyPath:/home/developer/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/developer/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/developer/.minikube} I0203 11:52:55.610257 31270 ubuntu.go:177] setting up certificates I0203 11:52:55.610265 31270 provision.go:83] configureAuth start I0203 11:52:55.610323 31270 cli_runner.go:133] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" minikube I0203 11:52:55.646447 31270 provision.go:138] copyHostCerts I0203 11:52:55.646517 31270 exec_runner.go:144] found /home/developer/.minikube/key.pem, removing ... I0203 11:52:55.646529 31270 exec_runner.go:207] rm: /home/developer/.minikube/key.pem I0203 11:52:55.646629 31270 exec_runner.go:151] cp: /home/developer/.minikube/certs/key.pem --> /home/developer/.minikube/key.pem (1679 bytes) I0203 11:52:55.646772 31270 exec_runner.go:144] found /home/developer/.minikube/ca.pem, removing ... I0203 11:52:55.646779 31270 exec_runner.go:207] rm: /home/developer/.minikube/ca.pem I0203 11:52:55.646824 31270 exec_runner.go:151] cp: /home/developer/.minikube/certs/ca.pem --> /home/developer/.minikube/ca.pem (1086 bytes) I0203 11:52:55.646979 31270 exec_runner.go:144] found /home/developer/.minikube/cert.pem, removing ... I0203 11:52:55.646986 31270 exec_runner.go:207] rm: /home/developer/.minikube/cert.pem I0203 11:52:55.647028 31270 exec_runner.go:151] cp: /home/developer/.minikube/certs/cert.pem --> /home/developer/.minikube/cert.pem (1131 bytes) I0203 11:52:55.647096 31270 provision.go:112] generating server cert: /home/developer/.minikube/machines/server.pem ca-key=/home/developer/.minikube/certs/ca.pem private-key=/home/developer/.minikube/certs/ca-key.pem org=developer.minikube san=[192.168.49.2 127.0.0.1 localhost 127.0.0.1 minikube minikube] I0203 11:52:55.739398 31270 provision.go:172] copyRemoteCerts I0203 11:52:55.739449 31270 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker I0203 11:52:55.739490 31270 cli_runner.go:133] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube I0203 11:52:55.779006 31270 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:49157 SSHKeyPath:/home/developer/.minikube/machines/minikube/id_rsa Username:docker} I0203 11:52:55.877876 31270 ssh_runner.go:362] scp /home/developer/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1086 bytes) I0203 11:52:55.898871 31270 ssh_runner.go:362] scp /home/developer/.minikube/machines/server.pem --> /etc/docker/server.pem (1208 bytes) I0203 11:52:55.920847 31270 ssh_runner.go:362] scp /home/developer/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1675 bytes) I0203 11:52:55.942173 31270 provision.go:86] duration metric: configureAuth took 331.895617ms I0203 11:52:55.942192 31270 ubuntu.go:193] setting minikube options for container-runtime I0203 11:52:55.942373 31270 config.go:176] Loaded profile config "minikube": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.23.1 I0203 11:52:55.942423 31270 cli_runner.go:133] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube I0203 11:52:55.980224 31270 main.go:130] libmachine: Using SSH client type: native I0203 11:52:55.980442 31270 main.go:130] libmachine: &{{{ 0 [] [] []} docker [0x7a0a80] 0x7a3b60 [] 0s} 127.0.0.1 49157 } I0203 11:52:55.980451 31270 main.go:130] libmachine: About to run SSH command: df --output=fstype / | tail -n 1 I0203 11:52:56.121696 31270 main.go:130] libmachine: SSH cmd err, output: : overlay

I0203 11:52:56.121709 31270 ubuntu.go:71] root file system type: overlay I0203 11:52:56.121903 31270 provision.go:309] Updating docker unit: /lib/systemd/system/docker.service ... I0203 11:52:56.121966 31270 cli_runner.go:133] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube I0203 11:52:56.159052 31270 main.go:130] libmachine: Using SSH client type: native I0203 11:52:56.159237 31270 main.go:130] libmachine: &{{{ 0 [] [] []} docker [0x7a0a80] 0x7a3b60 [] 0s} 127.0.0.1 49157 } I0203 11:52:56.159317 31270 main.go:130] libmachine: About to run SSH command: sudo mkdir -p /lib/systemd/system && printf %!s(MISSING) "[Unit] Description=Docker Application Container Engine Documentation=https://docs.docker.com BindsTo=containerd.service After=network-online.target firewalld.service containerd.service Wants=network-online.target Requires=docker.socket StartLimitBurst=3 StartLimitIntervalSec=60

[Service] Type=notify Restart=on-failure

This file is a systemd drop-in unit that inherits from the base dockerd configuration.

The base configuration already specifies an 'ExecStart=...' command. The first directive

here is to clear out that command inherited from the base configuration. Without this,

the command from the base configuration and the command specified here are treated as

a sequence of commands, which is not the desired behavior, nor is it valid -- systemd

will catch this invalid input and refuse to start the service with an error like:

Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.

NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other

container runtimes. If left unlimited, it may result in OOM issues with MySQL.

ExecStart= ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12 ExecReload=/bin/kill -s HUP \$MAINPID

Having non-zero Limit*s causes performance problems due to accounting overhead

in the kernel. We recommend using cgroups to do container-local accounting.

LimitNOFILE=infinity LimitNPROC=infinity LimitCORE=infinity

Uncomment TasksMax if your systemd version supports it.

Only systemd 226 and above support this version.

TasksMax=infinity TimeoutStartSec=0

set delegate yes so that systemd does not reset the cgroups of docker containers

Delegate=yes

kill only the docker process, not all processes in the cgroup

KillMode=process

[Install] WantedBy=multi-user.target " | sudo tee /lib/systemd/system/docker.service.new I0203 11:52:56.317434 31270 main.go:130] libmachine: SSH cmd err, output: : [Unit] Description=Docker Application Container Engine Documentation=https://docs.docker.com BindsTo=containerd.service After=network-online.target firewalld.service containerd.service Wants=network-online.target Requires=docker.socket StartLimitBurst=3 StartLimitIntervalSec=60

[Service] Type=notify Restart=on-failure

This file is a systemd drop-in unit that inherits from the base dockerd configuration.

The base configuration already specifies an 'ExecStart=...' command. The first directive

here is to clear out that command inherited from the base configuration. Without this,

the command from the base configuration and the command specified here are treated as

a sequence of commands, which is not the desired behavior, nor is it valid -- systemd

will catch this invalid input and refuse to start the service with an error like:

Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.

NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other

container runtimes. If left unlimited, it may result in OOM issues with MySQL.

ExecStart= ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12 ExecReload=/bin/kill -s HUP $MAINPID

Having non-zero Limit*s causes performance problems due to accounting overhead

in the kernel. We recommend using cgroups to do container-local accounting.

LimitNOFILE=infinity LimitNPROC=infinity LimitCORE=infinity

Uncomment TasksMax if your systemd version supports it.

Only systemd 226 and above support this version.

TasksMax=infinity TimeoutStartSec=0

set delegate yes so that systemd does not reset the cgroups of docker containers

Delegate=yes

kill only the docker process, not all processes in the cgroup

KillMode=process

[Install] WantedBy=multi-user.target

I0203 11:52:56.317513 31270 cli_runner.go:133] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube I0203 11:52:56.357154 31270 main.go:130] libmachine: Using SSH client type: native I0203 11:52:56.357324 31270 main.go:130] libmachine: &{{{ 0 [] [] []} docker [0x7a0a80] 0x7a3b60 [] 0s} 127.0.0.1 49157 } I0203 11:52:56.357342 31270 main.go:130] libmachine: About to run SSH command: sudo diff -u /lib/systemd/system/docker.service /lib/systemd/system/docker.service.new || { sudo mv /lib/systemd/system/docker.service.new /lib/systemd/system/docker.service; sudo systemctl -f daemon-reload && sudo systemctl -f enable docker && sudo systemctl -f restart docker; } I0203 11:52:56.507498 31270 main.go:130] libmachine: SSH cmd err, output: : I0203 11:52:56.507517 31270 machine.go:91] provisioned docker machine in 1.268416591s I0203 11:52:56.507527 31270 start.go:267] post-start starting for "minikube" (driver="docker") I0203 11:52:56.507534 31270 start.go:277] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs] I0203 11:52:56.507612 31270 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs I0203 11:52:56.507656 31270 cli_runner.go:133] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube I0203 11:52:56.557915 31270 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:49157 SSHKeyPath:/home/developer/.minikube/machines/minikube/id_rsa Username:docker} I0203 11:52:56.661892 31270 ssh_runner.go:195] Run: cat /etc/os-release I0203 11:52:56.665004 31270 main.go:130] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found I0203 11:52:56.665020 31270 main.go:130] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found I0203 11:52:56.665029 31270 main.go:130] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found I0203 11:52:56.665034 31270 info.go:137] Remote host: Ubuntu 20.04.2 LTS I0203 11:52:56.665042 31270 filesync.go:126] Scanning /home/developer/.minikube/addons for local assets ... I0203 11:52:56.665096 31270 filesync.go:126] Scanning /home/developer/.minikube/files for local assets ... I0203 11:52:56.665115 31270 start.go:270] post-start completed in 157.581551ms I0203 11:52:56.665149 31270 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'" I0203 11:52:56.665186 31270 cli_runner.go:133] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube I0203 11:52:56.702724 31270 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:49157 SSHKeyPath:/home/developer/.minikube/machines/minikube/id_rsa Username:docker} I0203 11:52:56.798021 31270 fix.go:57] fixHost completed within 1.600286693s I0203 11:52:56.798039 31270 start.go:80] releasing machines lock for "minikube", held for 1.600325697s I0203 11:52:56.798204 31270 cli_runner.go:133] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" minikube I0203 11:52:56.838124 31270 ssh_runner.go:195] Run: systemctl --version I0203 11:52:56.838154 31270 ssh_runner.go:195] Run: curl -sS -m 2 https://k8s.gcr.io/ I0203 11:52:56.838180 31270 cli_runner.go:133] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube I0203 11:52:56.838224 31270 cli_runner.go:133] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube I0203 11:52:56.881458 31270 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:49157 SSHKeyPath:/home/developer/.minikube/machines/minikube/id_rsa Username:docker} I0203 11:52:56.885409 31270 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:49157 SSHKeyPath:/home/developer/.minikube/machines/minikube/id_rsa Username:docker} I0203 11:52:56.977728 31270 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service containerd I0203 11:52:57.305832 31270 ssh_runner.go:195] Run: sudo systemctl cat docker.service I0203 11:52:57.319082 31270 cruntime.go:272] skipping containerd shutdown because we are bound to it I0203 11:52:57.319137 31270 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio I0203 11:52:57.331628 31270 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %!s(MISSING) "runtime-endpoint: unix:///var/run/dockershim.sock image-endpoint: unix:///var/run/dockershim.sock " | sudo tee /etc/crictl.yaml" I0203 11:52:57.349330 31270 ssh_runner.go:195] Run: sudo systemctl unmask docker.service I0203 11:52:57.457594 31270 ssh_runner.go:195] Run: sudo systemctl enable docker.socket I0203 11:52:57.566263 31270 ssh_runner.go:195] Run: sudo systemctl cat docker.service I0203 11:52:57.578768 31270 ssh_runner.go:195] Run: sudo systemctl daemon-reload I0203 11:52:57.688150 31270 ssh_runner.go:195] Run: sudo systemctl start docker I0203 11:52:57.703339 31270 ssh_runner.go:195] Run: docker version --format {{.Server.Version}} I0203 11:52:57.762815 31270 ssh_runner.go:195] Run: docker version --format {{.Server.Version}} I0203 11:52:57.820836 31270 out.go:203] * Preparing Kubernetes v1.23.1 on Docker 20.10.12 ... I0203 11:52:57.820962 31270 cli_runner.go:133] Run: docker network inspect minikube --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}" I0203 11:52:57.859318 31270 ssh_runner.go:195] Run: grep 192.168.49.1 host.minikube.internal$ /etc/hosts I0203 11:52:57.865597 31270 out.go:176] - kubelet.housekeeping-interval=5m I0203 11:52:57.866012 31270 preload.go:132] Checking if preload exists for k8s version v1.23.1 and runtime docker I0203 11:52:57.866058 31270 ssh_runner.go:195] Run: docker images --format {{.Repository}}:{{.Tag}} I0203 11:52:57.904965 31270 docker.go:606] Got preloaded images: -- stdout -- kubernetesui/dashboard:v2.3.1 kubernetesui/metrics-scraper:v1.0.7

-- /stdout -- I0203 11:52:57.904980 31270 docker.go:612] k8s.gcr.io/kube-apiserver:v1.23.1 wasn't preloaded I0203 11:52:57.904986 31270 cache_images.go:88] LoadImages start: [k8s.gcr.io/kube-apiserver:v1.23.1 k8s.gcr.io/kube-controller-manager:v1.23.1 k8s.gcr.io/kube-scheduler:v1.23.1 k8s.gcr.io/kube-proxy:v1.23.1 k8s.gcr.io/pause:3.6 k8s.gcr.io/etcd:3.5.1-0 k8s.gcr.io/coredns/coredns:v1.8.6 gcr.io/k8s-minikube/storage-provisioner:v5 docker.io/kubernetesui/dashboard:v2.3.1 docker.io/kubernetesui/metrics-scraper:v1.0.7] I0203 11:52:57.906167 31270 image.go:134] retrieving image: k8s.gcr.io/kube-apiserver:v1.23.1 I0203 11:52:57.906286 31270 image.go:134] retrieving image: docker.io/kubernetesui/metrics-scraper:v1.0.7 I0203 11:52:57.906744 31270 image.go:134] retrieving image: k8s.gcr.io/kube-scheduler:v1.23.1 I0203 11:52:57.906779 31270 image.go:134] retrieving image: k8s.gcr.io/kube-controller-manager:v1.23.1 I0203 11:52:57.906842 31270 image.go:134] retrieving image: k8s.gcr.io/etcd:3.5.1-0 I0203 11:52:57.907002 31270 image.go:134] retrieving image: k8s.gcr.io/pause:3.6 I0203 11:52:57.907010 31270 image.go:134] retrieving image: k8s.gcr.io/kube-proxy:v1.23.1 I0203 11:52:57.907130 31270 image.go:180] daemon lookup for docker.io/kubernetesui/metrics-scraper:v1.0.7: Error response from daemon: reference does not exist I0203 11:52:57.907134 31270 image.go:180] daemon lookup for k8s.gcr.io/kube-apiserver:v1.23.1: Error response from daemon: reference does not exist I0203 11:52:57.907172 31270 image.go:180] daemon lookup for k8s.gcr.io/kube-controller-manager:v1.23.1: Error response from daemon: reference does not exist I0203 11:52:57.907340 31270 image.go:134] retrieving image: gcr.io/k8s-minikube/storage-provisioner:v5 I0203 11:52:57.907434 31270 image.go:134] retrieving image: k8s.gcr.io/coredns/coredns:v1.8.6 I0203 11:52:57.907447 31270 image.go:134] retrieving image: docker.io/kubernetesui/dashboard:v2.3.1 I0203 11:52:57.907649 31270 image.go:180] daemon lookup for k8s.gcr.io/pause:3.6: Error response from daemon: reference does not exist I0203 11:52:57.907763 31270 image.go:180] daemon lookup for k8s.gcr.io/kube-proxy:v1.23.1: Error response from daemon: reference does not exist I0203 11:52:57.907901 31270 image.go:180] daemon lookup for k8s.gcr.io/etcd:3.5.1-0: Error response from daemon: reference does not exist I0203 11:52:57.908134 31270 image.go:180] daemon lookup for k8s.gcr.io/coredns/coredns:v1.8.6: Error response from daemon: reference does not exist I0203 11:52:57.908135 31270 image.go:180] daemon lookup for gcr.io/k8s-minikube/storage-provisioner:v5: Error response from daemon: reference does not exist I0203 11:52:57.908195 31270 image.go:180] daemon lookup for docker.io/kubernetesui/dashboard:v2.3.1: Error response from daemon: reference does not exist I0203 11:52:57.908301 31270 image.go:180] daemon lookup for k8s.gcr.io/kube-scheduler:v1.23.1: Error response from daemon: reference does not exist W0203 11:52:58.033719 31270 image.go:190] authn lookup for gcr.io/k8s-minikube/storage-provisioner:v5 (trying anon): Get "https://gcr.io/v2/": x509: certificate signed by unknown authority I0203 11:52:58.160251 31270 image.go:194] remote lookup for gcr.io/k8s-minikube/storage-provisioner:v5: Get "https://gcr.io/v2/": x509: certificate signed by unknown authority I0203 11:52:58.160264 31270 image.go:93] error retrieve Image gcr.io/k8s-minikube/storage-provisioner:v5 ref Error response from daemon: reference does not exist I0203 11:52:58.160280 31270 cache_images.go:116] "gcr.io/k8s-minikube/storage-provisioner:v5" needs transfer: got empty img digest "" for gcr.io/k8s-minikube/storage-provisioner:v5 I0203 11:52:58.160318 31270 docker.go:287] Removing image: gcr.io/k8s-minikube/storage-provisioner:v5 I0203 11:52:58.160358 31270 ssh_runner.go:195] Run: docker rmi gcr.io/k8s-minikube/storage-provisioner:v5 I0203 11:52:58.209378 31270 cache_images.go:286] Loading image from: /home/developer/.minikube/cache/images/gcr.io/k8s-minikube/storage-provisioner_v5 W0203 11:52:59.047340 31270 image.go:261] failed to get config for k8s.gcr.io/pause:3.6: Get "https://storage.googleapis.com/asia.artifacts.k8s-artifacts-prod.appspot.com/containers/images/sha256:6270bb605e12e581514ada5fd5b3216f727db55dc87d5889c790e4c760683fee": x509: certificate signed by unknown authority I0203 11:52:59.047362 31270 image.go:93] error retrieve Image k8s.gcr.io/pause:3.6 ref Error response from daemon: reference does not exist I0203 11:52:59.047388 31270 cache_images.go:116] "k8s.gcr.io/pause:3.6" needs transfer: got empty img digest "" for k8s.gcr.io/pause:3.6 I0203 11:52:59.047414 31270 docker.go:287] Removing image: k8s.gcr.io/pause:3.6 I0203 11:52:59.047477 31270 ssh_runner.go:195] Run: docker rmi k8s.gcr.io/pause:3.6 W0203 11:52:59.048387 31270 image.go:261] failed to get config for k8s.gcr.io/kube-proxy:v1.23.1: Get "https://storage.googleapis.com/asia.artifacts.k8s-artifacts-prod.appspot.com/containers/images/sha256:b46c42588d5116766d0eb259ff372e7c1e3ecc41a842b0c18a8842083e34d62e": x509: certificate signed by unknown authority I0203 11:52:59.048416 31270 image.go:93] error retrieve Image k8s.gcr.io/kube-proxy:v1.23.1 ref Error response from daemon: reference does not exist I0203 11:52:59.048433 31270 cache_images.go:116] "k8s.gcr.io/kube-proxy:v1.23.1" needs transfer: got empty img digest "" for k8s.gcr.io/kube-proxy:v1.23.1 I0203 11:52:59.048458 31270 docker.go:287] Removing image: k8s.gcr.io/kube-proxy:v1.23.1 I0203 11:52:59.048486 31270 ssh_runner.go:195] Run: docker rmi k8s.gcr.io/kube-proxy:v1.23.1 I0203 11:52:59.094791 31270 cache_images.go:286] Loading image from: /home/developer/.minikube/cache/images/k8s.gcr.io/kube-proxy_v1.23.1 I0203 11:52:59.094791 31270 cache_images.go:286] Loading image from: /home/developer/.minikube/cache/images/k8s.gcr.io/pause_3.6 W0203 11:52:59.117949 31270 image.go:261] failed to get config for k8s.gcr.io/etcd:3.5.1-0: Get "https://storage.googleapis.com/asia.artifacts.k8s-artifacts-prod.appspot.com/containers/images/sha256:25f8c7f3da61c2a810effe5fa779cf80ca171afb0adf94c7cb51eb9a8546629d": x509: certificate signed by unknown authority I0203 11:52:59.117986 31270 image.go:93] error retrieve Image k8s.gcr.io/etcd:3.5.1-0 ref Error response from daemon: reference does not exist I0203 11:52:59.118005 31270 cache_images.go:116] "k8s.gcr.io/etcd:3.5.1-0" needs transfer: got empty img digest "" for k8s.gcr.io/etcd:3.5.1-0 I0203 11:52:59.118045 31270 docker.go:287] Removing image: k8s.gcr.io/etcd:3.5.1-0 I0203 11:52:59.118085 31270 ssh_runner.go:195] Run: docker rmi k8s.gcr.io/etcd:3.5.1-0 W0203 11:52:59.118867 31270 image.go:261] failed to get config for k8s.gcr.io/kube-apiserver:v1.23.1: Get "https://storage.googleapis.com/asia.artifacts.k8s-artifacts-prod.appspot.com/containers/images/sha256:b6d7abedde39968d56e9f53aaeea02a4fe6413497c4dedf091868eae09dcc320": x509: certificate signed by unknown authority I0203 11:52:59.118886 31270 image.go:93] error retrieve Image k8s.gcr.io/kube-apiserver:v1.23.1 ref Error response from daemon: reference does not exist I0203 11:52:59.118905 31270 cache_images.go:116] "k8s.gcr.io/kube-apiserver:v1.23.1" needs transfer: got empty img digest "" for k8s.gcr.io/kube-apiserver:v1.23.1 I0203 11:52:59.118925 31270 docker.go:287] Removing image: k8s.gcr.io/kube-apiserver:v1.23.1 I0203 11:52:59.118956 31270 ssh_runner.go:195] Run: docker rmi k8s.gcr.io/kube-apiserver:v1.23.1 W0203 11:52:59.119818 31270 image.go:261] failed to get config for k8s.gcr.io/kube-controller-manager:v1.23.1: Get "https://storage.googleapis.com/asia.artifacts.k8s-artifacts-prod.appspot.com/containers/images/sha256:f51846a4fd28801f333d9a13e4a77a96bd52f06e587ba664c2914f015c38e5d1": x509: certificate signed by unknown authority I0203 11:52:59.119832 31270 image.go:93] error retrieve Image k8s.gcr.io/kube-controller-manager:v1.23.1 ref Error response from daemon: reference does not exist I0203 11:52:59.119842 31270 cache_images.go:116] "k8s.gcr.io/kube-controller-manager:v1.23.1" needs transfer: got empty img digest "" for k8s.gcr.io/kube-controller-manager:v1.23.1 I0203 11:52:59.119861 31270 docker.go:287] Removing image: k8s.gcr.io/kube-controller-manager:v1.23.1 I0203 11:52:59.119909 31270 ssh_runner.go:195] Run: docker rmi k8s.gcr.io/kube-controller-manager:v1.23.1 W0203 11:52:59.122681 31270 image.go:261] failed to get config for k8s.gcr.io/coredns/coredns:v1.8.6: Get "https://storage.googleapis.com/asia.artifacts.k8s-artifacts-prod.appspot.com/containers/images/sha256:a4ca41631cc7ac19ce1be3ebf0314ac5f47af7c711f17066006db82ee3b75b03": x509: certificate signed by unknown authority I0203 11:52:59.122698 31270 image.go:93] error retrieve Image k8s.gcr.io/coredns/coredns:v1.8.6 ref Error response from daemon: reference does not exist I0203 11:52:59.122708 31270 cache_images.go:116] "k8s.gcr.io/coredns/coredns:v1.8.6" needs transfer: got empty img digest "" for k8s.gcr.io/coredns/coredns:v1.8.6 I0203 11:52:59.122725 31270 docker.go:287] Removing image: k8s.gcr.io/coredns/coredns:v1.8.6 I0203 11:52:59.122756 31270 ssh_runner.go:195] Run: docker rmi k8s.gcr.io/coredns/coredns:v1.8.6 W0203 11:52:59.128787 31270 image.go:261] failed to get config for k8s.gcr.io/kube-scheduler:v1.23.1: Get "https://storage.googleapis.com/asia.artifacts.k8s-artifacts-prod.appspot.com/containers/images/sha256:71d575efe62835f4882115d409a676dd24102215eee650bf23b9cf42af0e7c05": x509: certificate signed by unknown authority I0203 11:52:59.128815 31270 image.go:93] error retrieve Image k8s.gcr.io/kube-scheduler:v1.23.1 ref Error response from daemon: reference does not exist I0203 11:52:59.128835 31270 cache_images.go:116] "k8s.gcr.io/kube-scheduler:v1.23.1" needs transfer: got empty img digest "" for k8s.gcr.io/kube-scheduler:v1.23.1 I0203 11:52:59.128866 31270 docker.go:287] Removing image: k8s.gcr.io/kube-scheduler:v1.23.1 I0203 11:52:59.128904 31270 ssh_runner.go:195] Run: docker rmi k8s.gcr.io/kube-scheduler:v1.23.1 I0203 11:52:59.190875 31270 cache_images.go:286] Loading image from: /home/developer/.minikube/cache/images/k8s.gcr.io/etcd_3.5.1-0 I0203 11:52:59.190917 31270 cache_images.go:286] Loading image from: /home/developer/.minikube/cache/images/k8s.gcr.io/kube-apiserver_v1.23.1 I0203 11:52:59.204034 31270 cache_images.go:286] Loading image from: /home/developer/.minikube/cache/images/k8s.gcr.io/kube-controller-manager_v1.23.1 I0203 11:52:59.206927 31270 cache_images.go:286] Loading image from: /home/developer/.minikube/cache/images/k8s.gcr.io/coredns/coredns_v1.8.6 I0203 11:52:59.222109 31270 cache_images.go:286] Loading image from: /home/developer/.minikube/cache/images/k8s.gcr.io/kube-scheduler_v1.23.1 I0203 11:53:01.307263 31270 ssh_runner.go:195] Run: docker image inspect --format {{.Id}} docker.io/kubernetesui/dashboard:v2.3.1 I0203 11:53:01.314907 31270 ssh_runner.go:195] Run: docker image inspect --format {{.Id}} docker.io/kubernetesui/metrics-scraper:v1.0.7 I0203 11:53:01.365005 31270 cache_images.go:92] LoadImages completed in 3.45998777s W0203 11:53:01.365197 31270 out.go:241] X Unable to load cached images: loading cached images: stat /home/developer/.minikube/cache/images/gcr.io/k8s-minikube/storage-provisioner_v5: no such file or directory I0203 11:53:01.365294 31270 ssh_runner.go:195] Run: docker info --format {{.CgroupDriver}} I0203 11:53:01.474659 31270 cni.go:93] Creating CNI manager for "" I0203 11:53:01.474671 31270 cni.go:167] CNI unnecessary in this configuration, recommending no CNI I0203 11:53:01.474689 31270 kubeadm.go:87] Using pod CIDR: 10.244.0.0/16 I0203 11:53:01.474703 31270 kubeadm.go:153] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.49.2 APIServerPort:8443 KubernetesVersion:v1.23.1 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:minikube NodeName:minikube DNSDomain:cluster.local CRISocket:/var/run/dockershim.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.49.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NoTaintMaster:true NodeIP:192.168.49.2 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt StaticPodPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[]} I0203 11:53:01.474917 31270 kubeadm.go:157] kubeadm config: apiVersion: kubeadm.k8s.io/v1beta2 kind: InitConfiguration localAPIEndpoint: advertiseAddress: 192.168.49.2 bindPort: 8443 bootstrapTokens:

I0203 11:53:01.475039 31270 kubeadm.go:791] kubelet [Unit] Wants=docker.socket

[Service] ExecStart= ExecStart=/var/lib/minikube/binaries/v1.23.1/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime=docker --hostname-override=minikube --housekeeping-interval=5m --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.49.2

[Install] config: {KubernetesVersion:v1.23.1 ClusterName:minikube Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: ExtraOptions:[{Component:kubelet Key:housekeeping-interval Value:5m}] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} I0203 11:53:01.475111 31270 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.23.1 I0203 11:53:01.484777 31270 binaries.go:47] Didn't find k8s binaries: didn't find preexisting kubelet Initiating transfer... I0203 11:53:01.484859 31270 ssh_runner.go:195] Run: sudo mkdir -p /var/lib/minikube/binaries/v1.23.1 I0203 11:53:01.494273 31270 download.go:101] Downloading: https://storage.googleapis.com/kubernetes-release/release/v1.23.1/bin/linux/amd64/kubelet?checksum=file:https://storage.googleapis.com/kubernetes-release/release/v1.23.1/bin/linux/amd64/kubelet.sha256 -> /home/developer/.minikube/cache/linux/v1.23.1/kubelet I0203 11:53:01.494276 31270 download.go:101] Downloading: https://storage.googleapis.com/kubernetes-release/release/v1.23.1/bin/linux/amd64/kubeadm?checksum=file:https://storage.googleapis.com/kubernetes-release/release/v1.23.1/bin/linux/amd64/kubeadm.sha256 -> /home/developer/.minikube/cache/linux/v1.23.1/kubeadm I0203 11:53:01.494881 31270 download.go:101] Downloading: https://storage.googleapis.com/kubernetes-release/release/v1.23.1/bin/linux/amd64/kubectl?checksum=file:https://storage.googleapis.com/kubernetes-release/release/v1.23.1/bin/linux/amd64/kubectl.sha256 -> /home/developer/.minikube/cache/linux/v1.23.1/kubectl I0203 11:53:01.702543 31270 out.go:176] W0203 11:53:01.702908 31270 out.go:241] X Exiting due to K8S_INSTALL_FAILED: updating control plane: downloading binaries: downloading kubeadm: download failed: https://storage.googleapis.com/kubernetes-release/release/v1.23.1/bin/linux/amd64/kubeadm?checksum=file:https://storage.googleapis.com/kubernetes-release/release/v1.23.1/bin/linux/amd64/kubeadm.sha256: getter: &{Ctx:context.Background Src:https://storage.googleapis.com/kubernetes-release/release/v1.23.1/bin/linux/amd64/kubeadm?checksum=file:https://storage.googleapis.com/kubernetes-release/release/v1.23.1/bin/linux/amd64/kubeadm.sha256 Dst:/home/developer/.minikube/cache/linux/v1.23.1/kubeadm.download Pwd: Mode:2 Umask:---------- Detectors:[0x358be48 0x358be48 0x358be48 0x358be48 0x358be48 0x358be48 0x358be48] Decompressors:map[bz2:0x358be48 gz:0x358be48 tar:0x358be48 tar.bz2:0x358be48 tar.gz:0x358be48 tar.xz:0x358be48 tar.zst:0x358be48 tbz2:0x358be48 tgz:0x358be48 txz:0x358be48 tzst:0x358be48 xz:0x358be48 zip:0x358be48 zst:0x358be48] Getters:map[file:0xc0005b1e30 http:0xc000580960 https:0xc0005809c0] Dir:false ProgressListener:0x354c130 Insecure:false Options:[0x18a59e0]}: invalid checksum: Error downloading checksum file: Get "https://storage.googleapis.com/kubernetes-release/release/v1.23.1/bin/linux/amd64/kubeadm.sha256": x509: certificate signed by unknown authority W0203 11:53:01.702981 31270 out.go:241] W0203 11:53:01.704394 31270 out.go:241] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮ │ │ │ If the above advice does not help, please let us know: │ │ https://github.com/kubernetes/minikube/issues/new/choose │ │ │ │ * Please run minikube logs --file=logs.txt and attach logs.txt to the GitHub issue. │ │ │ ╰─────────────────────────────────────────────────────────────────────────────────────────────╯

Operating System

Ubuntu

Driver

Docker

RA489 commented 2 years ago

/kind support

draksha22 commented 2 years ago

/assign

klaases commented 2 years ago

Hi @draksha22, are you still actively working on this issue?

klaases commented 2 years ago

Hi @kumaranraj2895, could you please run minikube logs --file=logs.txt and attach logs.txt file to the GitHub issue?

It seems that the existing logs on this issue may have been formatted inconsistently.

kumaranraj2895 commented 2 years ago

It is working after reinstallation

Harikantipudi commented 2 years ago

@kumaranraj2895

Can you please elaborate on the resolution. It will help, since i am struggling with the issue and stuck since last 2 days (Weekend went for a Toss)

I am on K8's 1.21 and 1.22

Client Version: version.Info{Major:"1", Minor:"21+", GitVersion:"v1.21.2-13+d2965f0db10712", GitCommit:"d2965f0db1071203c6f5bc662c2827c71fc8b20d", GitTreeState:"clean", BuildDate:"2021-06-26T01:02:11Z", GoVersion:"go1.16.5", Compiler:"gc", Platform:"darwin/amd64"} Server Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.5", GitCommit:"5c99e2ac2ff9a3c549d9ca665e7bc05a3e18f07e", GitTreeState:"clean", BuildDate:"2021-12-16T08:32:32Z", GoVersion:"go1.16.12", Compiler:"gc", Platform:"linux/amd64

minikube version: v1.22.0 commit: a03fbcf166e6f74ef224d4a63be4277d017bb62e

My Kubernetes on Docker is running and I see all images on Docker Hub but when I am starting minikube it is failing . I tried everything i can from logs, downgrading and upgrading etc. Not sure if this is any proxy/permission issue. I am on a MAC

😄 minikube v1.22.0 on Darwin 12.3.1 ✨ Using the docker driver based on existing profile 👍 Starting control plane node minikube in cluster minikube 🚜 Pulling base image ... 🏃 Updating the running docker "minikube" container ... ❗ The image 'k8s.gcr.io/etcd:3.4.13-0' was not found; unable to add it to cache. ❗ The image 'k8s.gcr.io/kube-apiserver:v1.21.2' was not found; unable to add it to cache. ❗ The image 'k8s.gcr.io/kube-proxy:v1.21.2' was not found; unable to add it to cache. ❗ The image 'k8s.gcr.io/pause:3.4.1' was not found; unable to add it to cache. ❗ The image 'k8s.gcr.io/kube-scheduler:v1.21.2' was not found; unable to add it to cache. ❗ The image 'k8s.gcr.io/coredns/coredns:v1.8.0' was not found; unable to add it to cache. ❗ The image 'k8s.gcr.io/kube-controller-manager:v1.21.2' was not found; unable to add it to cache. ❗ The image 'gcr.io/k8s-minikube/storage-provisioner:v5' was not found; unable to add it to cache. 🐳 Preparing Kubernetes v1.21.2 on Docker 20.10.7 ... ❌ Unable to load cached images: loading cached images: stat /Users/AH64229/.minikube/cache/images/k8s.gcr.io/kube-apiserver_v1.21.2: no such file or directory

❌ Exiting due to K8S_INSTALL_FAILED: updating control plane: downloading binaries: downloading kubelet: download failed: https://storage.googleapis.com/kubernetes-release/release/v1.21.2/bin/linux/amd64/kubelet?checksum=file:https://storage.googleapis.com/kubernetes-release/release/v1.21.2/bin/linux/amd64/kubelet.sha256: getter: &{Ctx:context.Background Src:https://storage.googleapis.com/kubernetes-release/release/v1.21.2/bin/linux/amd64/kubelet?checksum=file:https://storage.googleapis.com/kubernetes-release/release/v1.21.2/bin/linux/amd64/kubelet.sha256 Dst:/Users/AH64229/.minikube/cache/linux/v1.21.2/kubelet.download Pwd: Mode:2 Umask:---------- Detectors:[0x41ad788 0x41ad788 0x41ad788 0x41ad788 0x41ad788 0x41ad788 0x41ad788] Decompressors:map[bz2:0x41ad788 gz:0x41ad788 tar:0x41ad788 tar.bz2:0x41ad788 tar.gz:0x41ad788 tar.xz:0x41ad788 tar.zst:0x41ad788 tbz2:0x41ad788 tgz:0x41ad788 txz:0x41ad788 tzst:0x41ad788 xz:0x41ad788 zip:0x41ad788 zst:0x41ad788] Getters:map[file:0xc00165db60 http:0xc0000a4160 https:0xc0000a41a0] Dir:false ProgressListener:0x4170b00 Insecure:false Options:[0x26345e0]}: invalid checksum: Error downloading checksum file: Get "https://storage.googleapis.com/kubernetes-release/release/v1.21.2/bin/linux/amd64/kubelet.sha256": EOF

╭────────────────────────────────────────────────────────────────────╮ │ │ │ 😿 If the above advice does not help, please let us know: │ │ 👉 https://github.com/kubernetes/minikube/issues/new/choose │ │ │ │ Please attach the following file to the GitHub issue: │ │ 

            │

│

Appreiciate the inputs !!

kumaranraj2895 commented 2 years ago

@Harikantipudi

I had actually connected to a VPN, hence I was getting certificate issues, since the connection was not trusted. So I disconnected my VPN and re-installed minikube again

From your error message I see that it is not able to connect to k8s.gcr.io to fetch images. Try manually doing a docker pull on those images and see if the same issue replicates

Harikantipudi commented 2 years ago

@kumaranraj2895

I diconnected VPN and did try manually pulling earlier but in vain

Pulling from kube-controller-manager 5dea5ec2316d: Already exists 5ff0cfb5f412: Retrying in 1 second 523992968418: Retrying in 1 second error pulling image configuration: download failed after attempts=6: EOF

It says image already exists bcos this image is part of Kubernetes coming from Docker

I did reinstall minikube using both HomeBrew and manually but still issue persists . Not sure what is causing this. It could be very trivial but i may be missing something

kumaranraj2895 commented 2 years ago

@Harikantipudi Open a new issue, let's see if someone can provide a solution. Meanwhile try installing an older version of minikube.

mul14 commented 2 years ago

I also deal with same issue

$ minikube start
😄  minikube v1.25.2 on Ubuntu 22.04
🆕  Kubernetes 1.23.3 is now available. If you would like to upgrade, specify: --kubernetes-version=v1.23.3
✨  Using the docker driver based on existing profile
❗  docker is currently using the btrfs storage driver, consider switching to overlay2 for better performance
👍  Starting control plane node minikube in cluster minikube
🚜  Pulling base image ...
🏃  Updating the running docker "minikube" container ...
❗  The image 'k8s.gcr.io/kube-scheduler:v1.16.16' was not found; unable to add it to cache.
❗  The image 'k8s.gcr.io/kube-apiserver:v1.16.16' was not found; unable to add it to cache.
❗  The image 'k8s.gcr.io/kube-controller-manager:v1.16.16' was not found; unable to add it to cache.
❗  The image 'k8s.gcr.io/kube-proxy:v1.16.16' was not found; unable to add it to cache.
🐳  Preparing Kubernetes v1.16.16 on Docker 20.10.12 ...
    ▪ kubelet.housekeeping-interval=5m
❌  Unable to load cached images: loading cached images: stat /home/mul14/.minikube/cache/images/amd64/k8s.gcr.io/kube-apiserver_v1.16.16: no such file or directory

❌  Exiting due to K8S_INSTALL_FAILED: updating control plane: downloading binaries: downloading kubectl: download failed: https://storage.googleapis.com/kubernetes-release/release/v1.16.16/bin/linux/amd64/kubectl?checksum=file:https://storage.googleapis.com/kubernetes-release/release/v1.16.16/bin/linux/amd64/kubectl.sha1: getter: &{Ctx:context.Background Src:https://storage.googleapis.com/kubernetes-release/release/v1.16.16/bin/linux/amd64/kubectl?checksum=file:https://storage.googleapis.com/kubernetes-release/release/v1.16.16/bin/linux/amd64/kubectl.sha1 Dst:/home/mul14/.minikube/cache/linux/amd64/v1.16.16/kubectl.download Pwd: Mode:2 Umask:---------- Detectors:[0x36f58f8 0x36f58f8 0x36f58f8 0x36f58f8 0x36f58f8 0x36f58f8 0x36f58f8] Decompressors:map[bz2:0x36f58f8 gz:0x36f58f8 tar:0x36f58f8 tar.bz2:0x36f58f8 tar.gz:0x36f58f8 tar.xz:0x36f58f8 tar.zst:0x36f58f8 tbz2:0x36f58f8 tgz:0x36f58f8 txz:0x36f58f8 tzst:0x36f58f8 xz:0x36f58f8 zip:0x36f58f8 zst:0x36f58f8] Getters:map[file:0xc0010ffa00 http:0xc0004bd1c0 https:0xc0004bd1e0] Dir:false ProgressListener:0x36b4cc0 Insecure:false Options:[0x19449a0]}: invalid checksum: Error downloading checksum file: bad response code: 404

Just delete the ~/.minikube directory to solve that issue

$ rm -rf ~/.minikube 
$ minikube start    
😄  minikube v1.25.2 on Ubuntu 22.04
✨  Automatically selected the docker driver. Other choices: virtualbox, none, ssh
❗  docker is currently using the btrfs storage driver, consider switching to overlay2 for better performance
👍  Starting control plane node minikube in cluster minikube
🚜  Pulling base image ...
🔥  Creating docker container (CPUs=2, Memory=2200MB) ...
🐳  Preparing Kubernetes v1.23.3 on Docker 20.10.12 ...
    ▪ kubelet.housekeeping-interval=5m
    > kubectl.sha256: 64 B / 64 B [--------------------------] 100.00% ? p/s 0s
    > kubelet.sha256: 64 B / 64 B [--------------------------] 100.00% ? p/s 0s
    > kubeadm.sha256: 64 B / 64 B [--------------------------] 100.00% ? p/s 0s
    > kubectl: 44.43 MiB / 44.43 MiB [--------------] 100.00% 5.26 MiB p/s 8.6s
    > kubeadm: 43.12 MiB / 43.12 MiB [---------------] 100.00% 4.23 MiB p/s 10s
    > kubelet: 118.75 MiB / 118.75 MiB [---------] 100.00% 1008.34 KiB p/s 2m1s
    ▪ Generating certificates and keys ...
    ▪ Booting up control plane ...
    ▪ Configuring RBAC rules ...
🔎  Verifying Kubernetes components...
    ▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
🌟  Enabled addons: default-storageclass, storage-provisioner
🏄  Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default
chuan2019 commented 2 years ago

I encountered the same issue too. As @kumaranraj2895 mentioned, I tried starting the cluster with VPN and got the error. After disconnecting VPN, I got the cluster started successfully, even without reinstalling minikube or any extra action.

oluranticode commented 1 year ago

this solve my problem i used this "rm -rf ~/.minikube"

minnat65 commented 1 year ago

this solve my problem i used this "rm -rf ~/.minikube"

This worked for me.

ghost commented 1 year ago

I also deal with same issue

$ minikube start
😄  minikube v1.25.2 on Ubuntu 22.04
🆕  Kubernetes 1.23.3 is now available. If you would like to upgrade, specify: --kubernetes-version=v1.23.3
✨  Using the docker driver based on existing profile
❗  docker is currently using the btrfs storage driver, consider switching to overlay2 for better performance
👍  Starting control plane node minikube in cluster minikube
🚜  Pulling base image ...
🏃  Updating the running docker "minikube" container ...
❗  The image 'k8s.gcr.io/kube-scheduler:v1.16.16' was not found; unable to add it to cache.
❗  The image 'k8s.gcr.io/kube-apiserver:v1.16.16' was not found; unable to add it to cache.
❗  The image 'k8s.gcr.io/kube-controller-manager:v1.16.16' was not found; unable to add it to cache.
❗  The image 'k8s.gcr.io/kube-proxy:v1.16.16' was not found; unable to add it to cache.
🐳  Preparing Kubernetes v1.16.16 on Docker 20.10.12 ...
    ▪ kubelet.housekeeping-interval=5m
❌  Unable to load cached images: loading cached images: stat /home/mul14/.minikube/cache/images/amd64/k8s.gcr.io/kube-apiserver_v1.16.16: no such file or directory

❌  Exiting due to K8S_INSTALL_FAILED: updating control plane: downloading binaries: downloading kubectl: download failed: https://storage.googleapis.com/kubernetes-release/release/v1.16.16/bin/linux/amd64/kubectl?checksum=file:https://storage.googleapis.com/kubernetes-release/release/v1.16.16/bin/linux/amd64/kubectl.sha1: getter: &{Ctx:context.Background Src:https://storage.googleapis.com/kubernetes-release/release/v1.16.16/bin/linux/amd64/kubectl?checksum=file:https://storage.googleapis.com/kubernetes-release/release/v1.16.16/bin/linux/amd64/kubectl.sha1 Dst:/home/mul14/.minikube/cache/linux/amd64/v1.16.16/kubectl.download Pwd: Mode:2 Umask:---------- Detectors:[0x36f58f8 0x36f58f8 0x36f58f8 0x36f58f8 0x36f58f8 0x36f58f8 0x36f58f8] Decompressors:map[bz2:0x36f58f8 gz:0x36f58f8 tar:0x36f58f8 tar.bz2:0x36f58f8 tar.gz:0x36f58f8 tar.xz:0x36f58f8 tar.zst:0x36f58f8 tbz2:0x36f58f8 tgz:0x36f58f8 txz:0x36f58f8 tzst:0x36f58f8 xz:0x36f58f8 zip:0x36f58f8 zst:0x36f58f8] Getters:map[file:0xc0010ffa00 http:0xc0004bd1c0 https:0xc0004bd1e0] Dir:false ProgressListener:0x36b4cc0 Insecure:false Options:[0x19449a0]}: invalid checksum: Error downloading checksum file: bad response code: 404

Just delete the ~/.minikube directory to solve that issue

$ rm -rf ~/.minikube 
$ minikube start    
😄  minikube v1.25.2 on Ubuntu 22.04
✨  Automatically selected the docker driver. Other choices: virtualbox, none, ssh
❗  docker is currently using the btrfs storage driver, consider switching to overlay2 for better performance
👍  Starting control plane node minikube in cluster minikube
🚜  Pulling base image ...
🔥  Creating docker container (CPUs=2, Memory=2200MB) ...
🐳  Preparing Kubernetes v1.23.3 on Docker 20.10.12 ...
    ▪ kubelet.housekeeping-interval=5m
    > kubectl.sha256: 64 B / 64 B [--------------------------] 100.00% ? p/s 0s
    > kubelet.sha256: 64 B / 64 B [--------------------------] 100.00% ? p/s 0s
    > kubeadm.sha256: 64 B / 64 B [--------------------------] 100.00% ? p/s 0s
    > kubectl: 44.43 MiB / 44.43 MiB [--------------] 100.00% 5.26 MiB p/s 8.6s
    > kubeadm: 43.12 MiB / 43.12 MiB [---------------] 100.00% 4.23 MiB p/s 10s
    > kubelet: 118.75 MiB / 118.75 MiB [---------] 100.00% 1008.34 KiB p/s 2m1s
    ▪ Generating certificates and keys ...
    ▪ Booting up control plane ...
    ▪ Configuring RBAC rules ...
🔎  Verifying Kubernetes components...
    ▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
🌟  Enabled addons: default-storageclass, storage-provisioner
🏄  Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default

Worker for me too!