kubernetes / minikube

Run Kubernetes locally
https://minikube.sigs.k8s.io/
Apache License 2.0
28.89k stars 4.83k forks source link

Can not start minikube on MacOS #19014

Open obnoxxx opened 1 month ago

obnoxxx commented 1 month ago

What Happened?

I am unable to start minikube on a Mac with M3 processor, MacOS 14.5 for quite a while now. IInitially, t always reported a mix of driver and network issues.

3 days ago, I had some succes by explicitly specifying --driver qemu --network socket_vmnet, as specified on the website. I was able to create and start a minikube cluster once. But today, it fails again.

Here is the output:

1```console $ time minikube start 😄 minikube v1.33.1 on Darwin 14.5 (arm64) ✨ Automatically selected the qemu2 driver 🌐 Automatically selected the socket_vmnet network 👍 Starting "minikube" primary control-plane node in "minikube" cluster 🔥 Creating qemu2 VM (CPUs=2, Memory=6000MB, Disk=20000MB) ... 🔑 Your firewall is blocking bootpd which is required for socket_vmnet. The following commands will be executed to unblock bootpd:

$ sudo /usr/libexec/ApplicationFirewall/socketfilterfw --add /usr/libexec/bootpd 
$ sudo /usr/libexec/ApplicationFirewall/socketfilterfw --unblock /usr/libexec/bootpd 

🔄 Successfully unblocked bootpd process from firewall, retrying 🔥 Deleting "minikube" in qemu2 ... 🤦 StartHost failed, but will try again: creating host: create: creating: ip not found: failed to get IP address: could not find an IP address for 66:6d:2c:20:9:f3 🔥 Creating qemu2 VM (CPUs=2, Memory=6000MB, Disk=20000MB) ... 🔑 Your firewall is blocking bootpd which is required for socket_vmnet. The following commands will be executed to unblock bootpd:

$ sudo /usr/libexec/ApplicationFirewall/socketfilterfw --add /usr/libexec/bootpd 
$ sudo /usr/libexec/ApplicationFirewall/socketfilterfw --unblock /usr/libexec/bootpd 

🔄 Successfully unblocked bootpd process from firewall, retrying 😿 Failed to start qemu2 VM. Running "minikube delete" may fix it: creating host: create: creating: ip not found: failed to get IP address: could not find an IP address for 26:41:f9:70:7e:47

❌ Exiting due to GUEST_PROVISION: error provisioning guest: Failed to start host: creating host: create: creating: ip not found: failed to get IP address: could not find an IP address for 26:41:f9:70:7e:47 ╭───────────────────────────────────────────────────────────────────────────────────────────╮ │ │ │ 😿 If the above advice does not help, please let us know: │ │ 👉 https://github.com/kubernetes/minikube/issues/new/choose │ │ │ │ Please run minikube logs --file=logs.txt and attach logs.txt to the GitHub issue. │ │ │ ╰───────────────────────────────────────────────────────────────────────────────────────────╯

minikube start 0.74s user 0.97s system 0% cpu 4:10.61 total $


### Attach the log file

```console
$ inikube start

😄  minikube v1.33.1 on Darwin 14.5 (arm64)
✨  Automatically selected the qemu2 driver
🌐  Automatically selected the socket_vmnet network
👍  Starting "minikube" primary control-plane node in "minikube" cluster
🔥  Creating qemu2 VM (CPUs=2, Memory=6000MB, Disk=20000MB) ...
🔑  Your firewall is blocking bootpd which is required for socket_vmnet. The following commands will be executed to unblock bootpd:

    $ sudo /usr/libexec/ApplicationFirewall/socketfilterfw --add /usr/libexec/bootpd 
    $ sudo /usr/libexec/ApplicationFirewall/socketfilterfw --unblock /usr/libexec/bootpd 

🔄  Successfully unblocked bootpd process from firewall, retrying
🔥  Deleting "minikube" in qemu2 ...
🤦  StartHost failed, but will try again: creating host: create: creating: ip not found: failed to get IP address: could not find an IP address for 66:6d:2c:20:9:f3
🔥  Creating qemu2 VM (CPUs=2, Memory=6000MB, Disk=20000MB) ...
🔑  Your firewall is blocking bootpd which is required for socket_vmnet. The following commands will be executed to unblock bootpd:

    $ sudo /usr/libexec/ApplicationFirewall/socketfilterfw --add /usr/libexec/bootpd 
    $ sudo /usr/libexec/ApplicationFirewall/socketfilterfw --unblock /usr/libexec/bootpd 

🔄  Successfully unblocked bootpd process from firewall, retrying
😿  Failed to start qemu2 VM. Running "minikube delete" may fix it: creating host: create: creating: ip not found: failed to get IP address: could not find an IP address for 26:41:f9:70:7e:47

❌  Exiting due to GUEST_PROVISION: error provisioning guest: Failed to start host: creating host: create: creating: ip not found: failed to get IP address: could not find an IP address for 26:41:f9:70:7e:47
╭───────────────────────────────────────────────────────────────────────────────────────────╮
│                                                                                           │
│    😿  If the above advice does not help, please let us know:                             │
│    👉  https://github.com/kubernetes/minikube/issues/new/choose                           │
│                                                                                           │
│    Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.    │
│                                                                                           │
╰───────────────────────────────────────────────────────────────────────────────────────────╯
$ 

Operating System

macOS (Default)

Driver

QEMU

obnoxxx commented 1 month ago

@BlaineEXE , @raghavendra-talur : FYI

BlaineEXE commented 1 month ago

In my own experience, the issue is usually related to these firewalls, which I suspect because I see this output several times:

🔥 Creating qemu2 VM (CPUs=2, Memory=6000MB, Disk=20000MB) ... 🔑 Your firewall is blocking bootpd which is required for socket_vmnet. The following commands will be executed to unblock bootpd:

$ sudo /usr/libexec/ApplicationFirewall/socketfilterfw --add /usr/libexec/bootpd 
$ sudo /usr/libexec/ApplicationFirewall/socketfilterfw --unblock /usr/libexec/bootpd 

🔄 Successfully unblocked bootpd process from firewall, retrying 🔥 Deleting "minikube" in qemu2 ... 🤦 StartHost failed, but will try again: creating host: create: creating: ip not found: failed to get IP address: could not find an IP address for 66:6d:2c:20:9:f3

Have you checked the macos firewall to see whether those firewall entries are present? Here's what I see:

image

If not, try running the firewall commands manually maybe?

Also, you should verify that you've installed socket_vmnet via brew and enabled it via the brew options it recommends when installing. Notably, for me that has been to make sure to follow the step sudo brew services start socket_vmnet.

❯ brew info socket_vmnet                                     ✘ 255 3h 23m 18s
==> socket_vmnet: stable 1.1.4 (bottled), HEAD [keg-only]
Daemon to provide vmnet.framework support for rootless QEMU
https://github.com/lima-vm/socket_vmnet
Installed
/opt/homebrew/Cellar/socket_vmnet/1.1.2 (14 files, 120.7KB)
  Poured from bottle using the formulae.brew.sh API on 2023-05-31 at 09:55:31
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/s/socket_vmnet.rb
License: Apache-2.0
==> Requirements
Required: macOS ✔, macOS >= 10.15 (or Linux) ✔
==> Options
--HEAD
    Install HEAD version
==> Caveats
socket_vmnet requires root privileges so you will need to run
  `sudo /opt/homebrew/opt/socket_vmnet/socket_vmnet` or `sudo brew services start socket_vmnet`.
You should be certain that you trust any software you grant root privileges.

socket_vmnet is keg-only, which means it was not symlinked into /opt/homebrew,
because Homebrew's bin directory is often writable by a non-admin user.

To start socket_vmnet now and restart at startup:
  sudo brew services start socket_vmnet
Or, if you don't want/need a background service you can just run:
  /opt/homebrew/opt/socket_vmnet/bin/socket_vmnet --vmnet-gateway\=192.168.105.1 /opt/homebrew/var/run/socket_vmnet
obnoxxx commented 1 month ago

@BlaineEXE wrote:

In my own experience, the issue is usually related to these firewalls, which I suspect because I see this output several times:

🔥 Creating qemu2 VM (CPUs=2, Memory=6000MB, Disk=20000MB) ... 🔑 Your firewall is blocking bootpd which is required for socket_vmnet. The following commands will be executed to unblock bootpd:

$ sudo /usr/libexec/ApplicationFirewall/socketfilterfw --add /usr/libexec/bootpd 
$ sudo /usr/libexec/ApplicationFirewall/socketfilterfw --unblock /usr/libexec/bootpd 

🔄 Successfully unblocked bootpd process from firewall, retrying 🔥 Deleting "minikube" in qemu2 ... 🤦 StartHost failed, but will try again: creating host: create: creating: ip not found: failed to get IP address: could not find an IP address for 66:6d:2c:20:9:f3

Have you checked the macos firewall to see whether those firewall entries are present? Here's what I see:

image

looks the same for me. So it seems to have worked.

If not, try running the firewall commands manually maybe?

I had done that, but according to @raghavendra-talur , the main problem in my system seems to be that bootpdis not even running. And I don't know how to have it started.

Also, you should verify that you've installed socket_vmnet via brew and enabled it via the brew options it recommends when installing. Notably, for me that has been to make sure to follow the step sudo brew services start socket_vmnet.

I verified that socket_vmnet is installed (via brew) and it is really running (checked with ps.)

so currently I am stuck trying to figure out how to get bootpd started.

❯ brew info socket_vmnet                                     ✘ 255 3h 23m 18s
==> socket_vmnet: stable 1.1.4 (bottled), HEAD [keg-only]
Daemon to provide vmnet.framework support for rootless QEMU
https://github.com/lima-vm/socket_vmnet
Installed
/opt/homebrew/Cellar/socket_vmnet/1.1.2 (14 files, 120.7KB)
  Poured from bottle using the formulae.brew.sh API on 2023-05-31 at 09:55:31
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/s/socket_vmnet.rb
License: Apache-2.0
==> Requirements
Required: macOS ✔, macOS >= 10.15 (or Linux) ✔
==> Options
--HEAD
  Install HEAD version
==> Caveats
socket_vmnet requires root privileges so you will need to run
  `sudo /opt/homebrew/opt/socket_vmnet/socket_vmnet` or `sudo brew services start socket_vmnet`.
You should be certain that you trust any software you grant root privileges.

socket_vmnet is keg-only, which means it was not symlinked into /opt/homebrew,
because Homebrew's bin directory is often writable by a non-admin user.

To start socket_vmnet now and restart at startup:
  sudo brew services start socket_vmnet
Or, if you don't want/need a background service you can just run:
  /opt/homebrew/opt/socket_vmnet/bin/socket_vmnet --vmnet-gateway\=192.168.105.1 /opt/homebrew/var/run/socket_vmnet
obnoxxx commented 1 month ago

this might be a duplicate of #15960

At least this workaround now seems to have solved it for me: https://github.com/kubernetes/minikube/issues/15960#issuecomment-1641085186

obnoxxx commented 1 month ago

this might be a duplicate of #15960

At least this workaround now seems to have solved it for me: #15960 (comment)

Update: I am back to square one: Today, it dos not work for me anymore.

https://github.com/kubernetes/minikube/issues/15960#issuecomment-2149452613

how do I get bootpd running