ingress-dns image crash on macOS arm64

vkuznet commented 6 months ago

What Happened?

I'm using macOS (ARM64) with Docker Desktop and minikube. Once I deployed ingress-dns via the following command:

minikube addons enable ingress-dns

I see the following crash:

kube-system     kube-ingress-dns-minikube                   0/1     CrashLoopBackOff   6 (2m55s ago)   8m33s

If I look-up logs from this pod, I see wrong exec type:

kubectl logs -n kube-system kube-ingress-dns-minikube
exec /usr/local/bin/docker-entrypoint.sh: exec format error

Therefore, I have suspicion that ingress dns image does not contain proper command for my architecture.

Attach the log file

log.txt

Operating System

macOS (Default)

Driver

Docker

justinaslelys commented 6 months ago

I've got exactly the same problem on arm64 mac. Looks like the image for minikube-ingress-dns is built for single platform (amd64), You can see this yourself by running

docker manifest inspect gcr.io/k8s-minikube/minikube-ingress-dns:0.0.2@sha256:4abe27f9fc03fedab1d655e2020e6b165faf3bf6de1088ce6cf215a75b78f05f -v
{
    "Ref": "gcr.io/k8s-minikube/minikube-ingress-dns:0.0.2",
    "Descriptor": {
        "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
        "digest": "sha256:4abe27f9fc03fedab1d655e2020e6b165faf3bf6de1088ce6cf215a75b78f05f",
        "size": 1997,
        "platform": {
            "architecture": "amd64",
            "os": "linux"
        }
    },
    <..>

I am unsure if this repo is right place to report this because according to the links in docs ingress-dns image is managed by some cryptexlabs

vkuznet commented 6 months ago

For x-reference, I filed similar issue over here: https://gitlab.com/cryptexlabs/public/development/minikube-ingress-dns/-/issues/8

tdtu98 commented 5 months ago

What Happened?

I'm using macOS (ARM64) with Docker Desktop and minikube. Once I deployed ingress-dns via the following command:
minikube addons enable ingress-dns
I see the following crash:
kube-system     kube-ingress-dns-minikube                   0/1     CrashLoopBackOff   6 (2m55s ago)   8m33s
If I look-up logs from this pod, I see wrong exec type:
kubectl logs -n kube-system kube-ingress-dns-minikube
exec /usr/local/bin/docker-entrypoint.sh: exec format error
Therefore, I have suspicion that ingress dns image does not contain proper command for my architecture.

Attach the log file

log.txt

Operating System

macOS (Default)

Driver

Docker

You can try a workaround which is to use qemu-user-static for running the amd64 image of kube-ingress-dns-minikube.

minikube ssh "sudo apt-get update && sudo apt-get -y install qemu-user-static"
minikube stop
minikube start

emanuil-tolev commented 5 months ago

I've got the exact same problem, M1 Mac, same crashing pod and same output in the log as the original post. Weirdly enough, it was working in early February, I've had this arm chip laptop for 2 years, and we haven't changed our minikube config in the meantime. Did the image get rebuilt or something?

elja commented 5 months ago

having a similar issue, I'm on m1, qemy driver, minikube v1.32.0

vkuznet commented 5 months ago

Even though solution pointed by @tdtu98 seems to resolve the error in a pod I am unable to connect the minikube from my localhost. I tried many things and even though my DNS resolver knows about minikube IP I still out of lack to connect to in (ping does not work). Said that, I switched to docker desktop Kubernetes option and it seems to work. Here is a recipe I used (if someone else may need it):

Setup kubernetes cluster on macOS (M2, arm64)

install docker desktop
visit docker desktop settings and enable Kuberneres

install kubectl and configure it to use docker desktop

docker context ls
kubectl config use-context docker-desktop
kubectl describe node docker-desktop

setup nginx ingress

# please use recent version from
# https://github.com/kubernetes/ingress-nginx
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.10.0/deploy/static/provider/cloud/deploy.yaml

the docker desktop URL and IP can be found via:


kubectl cluster-info
Kubernetes control plane is running at https://kubernetes.docker.internal:6443
CoreDNS is running at https://kubernetes.docker.internal:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.


After these steps I was able to connect to my k8s cluster running within docker desktop just fine. Even though it still does not solve the reported issue I am happy with current setup via docker desktop and can proceed development.

bhundven commented 3 months ago

The solution from @tdtu98 no longer works. The newer gcr.io/k8s-minikube/kicbase images are not based on ubuntu anymore? (where can I find the source of this image?)

$ minikube ssh "sudo apt-get update && sudo apt-get -y install qemu-user-static"
sudo: apt-get: command not found
ssh: Process exited with status 1
$ minikube ssh
                         _             _
            _         _ ( )           ( )
  ___ ___  (_)  ___  (_)| |/')  _   _ | |_      __
/' _ ` _ `\| |/' _ `\| || , <  ( ) ( )| '_`\  /'__`\
| ( ) ( ) || || ( ) || || |\`\ | (_) || |_) )(  ___/
(_) (_) (_)(_)(_) (_)(_)(_) (_)`\___/'(_,__/'`\____)

$ cat /version.json
{"iso_version": "v1.33.1", "kicbase_version": "v0.0.43-1714992375-18804", "minikube_version": "v1.33.1", "commit": "d6e0d89dd5607476c1efbac5f05c928d4cd7ef53"}

There is a PR here to use a newer ingress-dns: https://github.com/kubernetes/minikube/pull/17926 But it's not done yet.

k8s-triage-robot commented 1 week ago

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue as fresh with /remove-lifecycle stale
Close this issue with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

kubernetes / minikube