search

docker / for-mac

Bug reports for Docker Desktop for Mac
https://www.docker.com/products/docker#/mac
2.44k stars 118 forks source link

Kubernetes cannot start because coredns image cannot be found #6703

Open ayarosh opened 1 year ago

ayarosh commented 1 year ago

Expected behavior

Kubernetes is up and running

Actual behavior

Kubernetes failed to start

Information

Output of /Applications/Docker.app/Contents/MacOS/com.docker.diagnose check

Steps to reproduce the behavior

The issue has been found and solved by myself.

CAUSE

The cause of the issue is that coredns cannot pull the image because the path is wrong. Kubelet error: Failed to pull image "k8s.gcr.io/coredns:v1.9.3": rpc error: code = Unknown desc = Error response from daemon: failed to resolve reference "k8s.gcr.io/coredns:v1.9.3": k8s.gcr.io/coredns:v1.9.3: not found

SOLUTION [PATCH IS REQUIRED]

Right after update the image path on "k8s.gcr.io/coredns/coredns:v1.9.3" the service has been started.

So, please, add this fix to the next release,

nicks commented 1 year ago

hmmm...i wasn't able to reproduce this. When I enabled kubernetes and checked the pods, I saw it pull registry.k8s.io/coredns/coredns:v1.9.3. can you post your diagnostic id? https://docs.docker.com/desktop/troubleshoot/overview/#diagnose

ayarosh commented 1 year ago

I've already fixed this on my laptop. As I have some apps that are running on my machine (elasticsearch, prometheus, e.t.c.) I was not tend to reset my kube cluster. So, I found that the coredns pods in kube-system namespace cannot pull the image and cannot start. So, I've manually updated the deployment config for coredns and the problem has gone. I'm not sure that if I will revert those changes it give the proper diagnostic info because it will be syntetic test. And I will definitely not tend to uninstall the Docker Desktop and drop the all services in my kube. So, you decide if this make sense to check and fix.

greenflute commented 1 year ago

Same Mac OS and Docker version, but with Intel Chip, after upgrade the kubernetes just won't start, had the kubernetes reseted several times, disabled und re-enabled several times, but didn't fix the problem.

rafrafek commented 1 year ago

coredns shows amd64 warning label for me. All other Kubernetes containers are m1 native.

wp4nuv commented 1 year ago

coredns shows amd64 warning label for me. All other Kubernetes containers are m1 native.

It does for me as well. I was researching about why that was when I found this discussion. I don't use coredns on docker for mac, but I would assume that amd64 translation will take a performance hit on Mac M1/M2 machines.

IAlexEgorov commented 1 year ago

I have the same issue Last time I saw label in the https://kubernetes.io "Legacy k8s.gcr.io container image registry will be redirected to registry.k8s.io k8s.gcr.io image registry will be redirected to registry.k8s.io on Monday March 20th."

Problem: Failed to pull image "k8s.gcr.io/coredns:v1.9.3"

I think, we need to change registry in settings or something like that

IAlexEgorov commented 1 year ago

I have changed k8s.gcr.io/coredns:v1.9.3 on registry.k8s.io/coredns/coredns:v1.9.3 in deployment/coredns and it solved my trouble, but I guess it doesn't best practice method

Startouf commented 1 year ago

For those k8s-beginners like me who struggled to figure out how to apply the patch

kubectl -n kube-system set image deployment/coredns coredns=k8s.gcr.io/coredns/coredns:v1.9.3
greenflute commented 1 year ago

For those k8s-beginners like me who struggled to figure out how to apply the patch

kubectl -n kube-system set image deployment/coredns coredns=k8s.gcr.io/coredns/coredns:v1.9.3

Thanks for the tip, but doesn't work on my intel mbp:

% nc -zv kubernetes.docker.internal 6443
Connection to kubernetes.docker.internal port 6443 [tcp/sun-sr-https] succeeded!

% kubectl -n kube-system set image deployment/coredns coredns=k8s.gcr.io/coredns/coredns:v1.9.3
Unable to connect to the server: EOF
NimzyMaina commented 1 year ago 
$ docker pull k8s.gcr.io/coredns/coredns:v1.9.3
$ docker tag k8s.gcr.io/coredns/coredns:v1.9.3 k8s.gcr.io/coredns:v1.9.3
greenflute commented 1 year ago

` % docker pull k8s.gcr.io/coredns/coredns:v1.9.3
v1.9.3: Pulling from coredns/coredns Digest: sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a Status: Downloaded newer image for k8s.gcr.io/coredns/coredns:v1.9.3 k8s.gcr.io/coredns/coredns:v1.9.3

What's Next? View summary of image vulnerabilities and recommendations → docker scout quickview k8s.gcr.io/coredns/coredns:v1.9.3 % docker tag k8s.gcr.io/coredns/coredns:v1.9.3 k8s.gcr.io/coredns:v1.9.3

`

What a magic, the k8s now starts like a charm, sincerely thanks