search

kubernetes / minikube

Run Kubernetes locally
https://minikube.sigs.k8s.io/
Apache License 2.0
29.14k stars 4.86k forks source link

Exiting Due To Guest Start when installing Minikube on Colima x86_64 on Mac M1 #18258

Closed adamlwatson closed 1 month ago

adamlwatson commented 6 months ago

What Happened?

I'm trying to install Minikube on a Colima x86_64 instalnce on a Mac M1. I'm starting the Colima instance using this configuration:

colima start --cpu 4 --memory 10 --disk 100 --runtime docker --arch x86_64 --vm-type vz Once the Colima instance is running and I verify the Docker Daemon is running, I then try to start Minikube using this configuration:

minikube start --cpus 4 --memory 9G --disk-size 90G --driver docker --container-runtime docker

Toward the end of setup, I receive an error:

❌  Exiting due to GUEST_START: failed to start node: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.28.3:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml  --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 2
stdout:

stderr:
/var/lib/minikube/binaries/v1.28.3/kubeadm: 2: Syntax error: newline unexpected (expecting ")")

I've seen another issue that was similar to this, and have tried a minikube delete --all --purge to no avail.

I am able to spin up amd64 images under the Docker instance running on Colima x86_64 manually.

If I start minikube under a Colima instance that's set to the aarch64 archirtecture with Rosetta enabbled, the minikube installation works as expected... but then I can't seem to get any linux/amd64 images to run (of course).

Attach the log file

logs.txt

Operating System

macOS (Default)

Driver

Docker

alexandrpaliy commented 6 months ago

Same issue here.

$ sw_vers
ProductName:        macOS
ProductVersion:     14.3.1
BuildVersion:       23D60

$ colima --version
colima version 0.6.8

$ minikube version                                                                                                          14
minikube version: v1.32.0
commit: 8220a6eb95f0a4d75f7f2d7b14cef975f050512d
$ cat ~/.colima/default/colima.yaml | grep -Ev '(^\s*#)|(^\s*$)'
cpu: 4
disk: 60
memory: 5
arch: x86_64
runtime: docker
hostname: colima
kubernetes:
  enabled: false
  version: v1.28.3+k3s2
  k3sArgs:
    - --disable=traefik
autoActivate: true
network:
  address: true
  dns: []
  dnsHosts: {}
forwardAgent: false
docker: {}
vmType: vz
rosetta: true
mountType: sshfs
mountInotify: true
cpuType: ""
provision: []
sshConfig: true
mounts: []
env: {}

minikube start log: https://gist.github.com/alexandrpaliy/3fecf00527b0b512267c06694d8be7d9

alexandrpaliy commented 6 months ago

I think this deserves explicit message instead of editing my previous post.

Changing vmType from vz to qemu resolved the issue - at least, minikube seems up and running. According to https://lima-vm.io/docs/config/vmtype/#caveats :

Virtualization.framework doesn’t support running “intel guest on arm” and vice versa

Since I do indeed have arm CPU, and I tried to use arch: x86_64 in colima configuration - I think the error makes sense in the end. Maybe the text of the error doesn't, though :)

afbjorklund commented 6 months ago

Running minikube in emulation (in general) isn't supported, since it is too slow for most usages...

For containers it is better to run them with "fast mode", but node configuration in k8s gets tricky (how to configure a node that is of the arm64 architecture, but can run containers for amd64?)

https://lima-vm.io/docs/config/multi-arch/

Unfortunately the Kubernetes tutorials require it (also being able to run x86_64 containers) Because the example project* hasn't been updated for 5+ years, and thus isn't multi-arch yet

* https://kubernetes.io/docs/tutorials/kubernetes-basics/deploy-app/deploy-intro/

REPOSITORY                                  TAG       IMAGE ID       CREATED       SIZE
gcr.io/google-samples/kubernetes-bootcamp   v1        8fafd8af70e9   7 years ago   211MB

But for the minikube VM (start --vm), having hardware acceleration is considered mandatory.

afbjorklund commented 6 months ago
k8s-triage-robot commented 3 months ago

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot commented 2 months ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

k8s-triage-robot commented 1 month ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

k8s-ci-robot commented 1 month ago

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to [this](https://github.com/kubernetes/minikube/issues/18258#issuecomment-2267620042): >The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. > >This bot triages issues according to the following rules: >- After 90d of inactivity, `lifecycle/stale` is applied >- After 30d of inactivity since `lifecycle/stale` was applied, `lifecycle/rotten` is applied >- After 30d of inactivity since `lifecycle/rotten` was applied, the issue is closed > >You can: >- Reopen this issue with `/reopen` >- Mark this issue as fresh with `/remove-lifecycle rotten` >- Offer to help out with [Issue Triage][1] > >Please send feedback to sig-contributor-experience at [kubernetes/community](https://github.com/kubernetes/community). > >/close not-planned > >[1]: https://www.kubernetes.dev/docs/guide/issue-triage/ Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.