search

kubernetes / minikube

Run Kubernetes locally
https://minikube.sigs.k8s.io/
Apache License 2.0
28.87k stars 4.82k forks source link

Minikube tunnel seems to be broken with Parallels Driver #11229

Open pierrebeaucamp opened 3 years ago

pierrebeaucamp commented 3 years ago

As the title says, somehow minikube tunnel doesn't seem to work with the Parallels Driver. I'm using Minikube v1.19.0 (commit 15cede53bdc5fe242228853e737333b09d4336b5) installed through homebrew, and Parallels 16.5.0 (49183).

I don't have this issue when using hyperkit as the driver.

Steps to reproduce the issue:

  1. minikube start --driver=parallels
  2. minikube tunnel (in separate terminal)
  3. kubectl create deployment hello-minikube1 --image=k8s.gcr.io/echoserver:1.4
  4. kubectl expose deployment hello-minikube1 --type=LoadBalancer --port=8080
  5. curl -v <EXTERNAL-IP>:8080

Full output of failed command:

$ minikube tunnel
Password:
Status:
        machine: minikube
        pid: 55633
        route: 10.96.0.0/12 -> 10.211.55.10
        minikube: Running
        services: [hello-minikube1]
    errors: 
                minikube: no errors
                router: no errors
                loadbalancer emulator: no errors
$ curl -v 10.97.233.229:8080
*   Trying 10.97.233.229...
* TCP_NODELAY set
* Connection failed
* connect to 10.97.233.229 port 8080 failed: Operation timed out
* Failed to connect to 10.97.233.229 port 8080: Operation timed out
* Closing connection 0
curl: (7) Failed to connect to 10.97.233.229 port 8080: Operation timed out
Full output of `minikube start` command used: 
I0429 11:57:36.768303   54098 out.go:278] Setting OutFile to fd 1 ...
I0429 11:57:36.768594   54098 out.go:330] isatty.IsTerminal(1) = false
I0429 11:57:36.768635   54098 out.go:291] Setting ErrFile to fd 2...
I0429 11:57:36.768656   54098 out.go:330] isatty.IsTerminal(2) = false
I0429 11:57:36.768892   54098 root.go:317] Updating PATH: /Users/pierrebeaucamp/.minikube/bin
I0429 11:57:36.769462   54098 out.go:285] Setting JSON to false
I0429 11:57:36.803584   54098 start.go:108] hostinfo: {"hostname":"macbook-pierre.local","uptime":160497,"bootTime":1619551359,"procs":643,"os":"darwin","platform":"darwin","platformFamily":"Standalone Workstation","platformVersion":"11.3","kernelVersion":"20.4.0","kernelArch":"x86_64","virtualizationSystem":"","virtualizationRole":"","hostId":"bb2ffeba-7d53-301f-a238-9867ca51276f"}
W0429 11:57:36.803745   54098 start.go:116] gopshost.Virtualization returned error: not implemented yet
I0429 11:57:36.808505   54098 out.go:157] * minikube v1.19.0 on Darwin 11.3
* minikube v1.19.0 on Darwin 11.3
I0429 11:57:36.809631   54098 notify.go:126] Checking for updates...
I0429 11:57:36.809896   54098 driver.go:322] Setting default libvirt URI to qemu:///system
I0429 11:57:36.815407   54098 out.go:157] * Using the parallels driver based on user configuration
* Using the parallels driver based on user configuration
I0429 11:57:36.816482   54098 start.go:276] selected driver: parallels
I0429 11:57:36.816506   54098 start.go:718] validating driver "parallels" against 
I0429 11:57:36.816538   54098 start.go:729] status for parallels: {Installed:true Healthy:true Running:false NeedsImprovement:false Error: Reason: Fix: Doc:}
I0429 11:57:36.816769   54098 start_flags.go:253] no existing cluster config was found, will generate one from the flags 
I0429 11:57:36.819223   54098 start_flags.go:311] Using suggested 4000MB memory alloc based on sys=16384MB, container=0MB
I0429 11:57:36.819438   54098 start_flags.go:730] Wait components to verify : map[apiserver:true system_pods:true]
I0429 11:57:36.819488   54098 cni.go:81] Creating CNI manager for ""
I0429 11:57:36.819508   54098 cni.go:153] CNI unnecessary in this configuration, recommending no CNI
I0429 11:57:36.819524   54098 start_flags.go:270] config:
{Name:minikube KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.20@sha256:0250dab3644403384bd54f566921c6b57138eecffbb861f9392feef9b2ec44f6 Memory:4000 CPUs:2 DiskSize:20000 VMDriver: Driver:parallels HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.99.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.2 ClusterName:minikube Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[] Addons:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop: ExposedPorts:[] ListenAddress: Network: MultiNodeRequested:false}
I0429 11:57:36.819662   54098 iso.go:120] acquiring lock: {Name:mk9e6d25501940a7d4987aac3dda71e0603dd6c0 Clock:{} Delay:500ms Timeout:10m0s Cancel:}
I0429 11:57:36.822722   54098 out.go:157] * Starting control plane node minikube in cluster minikube
* Starting control plane node minikube in cluster minikube
I0429 11:57:36.823209   54098 preload.go:97] Checking if preload exists for k8s version v1.20.2 and runtime docker
I0429 11:57:36.823347   54098 preload.go:105] Found local preload: /Users/pierrebeaucamp/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v10-v1.20.2-docker-overlay2-amd64.tar.lz4
I0429 11:57:36.825735   54098 cache.go:54] Caching tarball of preloaded images
I0429 11:57:36.825809   54098 preload.go:131] Found /Users/pierrebeaucamp/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v10-v1.20.2-docker-overlay2-amd64.tar.lz4 in cache, skipping download
I0429 11:57:36.825992   54098 cache.go:57] Finished verifying existence of preloaded tar for  v1.20.2 on docker
I0429 11:57:36.826370   54098 profile.go:148] Saving config to /Users/pierrebeaucamp/.minikube/profiles/minikube/config.json ...
I0429 11:57:36.826437   54098 lock.go:36] WriteFile acquiring /Users/pierrebeaucamp/.minikube/profiles/minikube/config.json: {Name:mk5a62fd77ea0761500202be4d60e2ea1f0b51cc Clock:{} Delay:500ms Timeout:1m0s Cancel:}
I0429 11:57:36.826814   54098 cache.go:185] Successfully downloaded all kic artifacts
I0429 11:57:36.826864   54098 start.go:313] acquiring machines lock for minikube: {Name:mk94290631b223447da9672190052083ddc7a8fc Clock:{} Delay:500ms Timeout:13m0s Cancel:}
I0429 11:57:36.826979   54098 start.go:317] acquired machines lock for "minikube" in 97.938µs
I0429 11:57:36.827019   54098 start.go:89] Provisioning new machine with config: &{Name:minikube KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube/iso/minikube-v1.19.0.iso KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.20@sha256:0250dab3644403384bd54f566921c6b57138eecffbb861f9392feef9b2ec44f6 Memory:4000 CPUs:2 DiskSize:20000 VMDriver: Driver:parallels HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.99.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.2 ClusterName:minikube Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.20.2 ControlPlane:true Worker:true}] Addons:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop: ExposedPorts:[] ListenAddress: Network: MultiNodeRequested:false} &{Name: IP: Port:8443 KubernetesVersion:v1.20.2 ControlPlane:true Worker:true}
I0429 11:57:36.827093   54098 start.go:126] createHost starting for "" (driver="parallels")
I0429 11:57:36.831781   54098 out.go:184] * Creating parallels VM (CPUs=2, Memory=4000MB, Disk=20000MB) ...
* Creating parallels VM (CPUs=2, Memory=4000MB, Disk=20000MB) ...
I0429 11:57:36.832004   54098 start.go:160] libmachine.API.Create for "minikube" (driver="parallels")
I0429 11:57:36.832084   54098 client.go:168] LocalClient.Create starting
I0429 11:57:36.832538   54098 main.go:126] libmachine: Reading certificate data from /Users/pierrebeaucamp/.minikube/certs/ca.pem
I0429 11:57:36.832652   54098 main.go:126] libmachine: Decoding PEM data...
I0429 11:57:36.832692   54098 main.go:126] libmachine: Parsing certificate...
I0429 11:57:36.832923   54098 main.go:126] libmachine: Reading certificate data from /Users/pierrebeaucamp/.minikube/certs/cert.pem
I0429 11:57:36.833242   54098 main.go:126] libmachine: Decoding PEM data...
I0429 11:57:36.833267   54098 main.go:126] libmachine: Parsing certificate...
I0429 11:57:36.833356   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl --version
I0429 11:57:36.939736   54098 main.go:126] libmachine: executing: /usr/local/bin/prlsrvctl info --license
I0429 11:57:37.107789   54098 main.go:126] libmachine: Found Parallels Desktop version: &{%!d(string=) %!d(string=) [16 5 0] 3 %!d(string=16.5.0)}, edition: pro
I0429 11:57:37.107830   54098 main.go:126] libmachine: executing: /usr/local/bin/prlsrvctl net info Shared
I0429 11:57:37.372953   54098 main.go:126] libmachine: IP address of Shared network adapter: 10.211.55.2
I0429 11:57:37.373166   54098 main.go:126] libmachine: All host interface addressess: [127.0.0.1/8 ::1/128 fe80::1/64 fe80::492:f46b:ac9b:e424/64 10.88.111.9/24 fe80::ec47:5bff:fe81:3ff4/64 fe80::ec47:5bff:fe81:3ff4/64 fe80::5875:413d:9a9b:f75c/64 fe80::da33:133e:8ef3:2a46/64 fe80::f521:f26e:f6c2:232d/64 fe80::d69:140d:5c05:8a11/64 10.211.55.2/24 fe80::61:f218:1203:29a4/64 10.37.129.2/24 fe80::1c8c:87e4:b408:632c/64]
I0429 11:57:37.373182   54098 main.go:126] libmachine: Parallels Shared network adapter is connected
I0429 11:57:37.373772   54098 main.go:126] libmachine: Downloading /Users/pierrebeaucamp/.minikube/cache/boot2docker.iso from file:///Users/pierrebeaucamp/.minikube/cache/iso/minikube-v1.19.0.iso...
I0429 11:57:37.720919   54098 main.go:126] libmachine: Creating SSH key...
I0429 11:57:37.720962   54098 main.go:126] libmachine: SSH key: /Users/pierrebeaucamp/.minikube/machines/minikube/id_rsa
I0429 11:57:37.776454   54098 main.go:126] libmachine: Creating Parallels Desktop VM...
I0429 11:57:37.776477   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl --version
I0429 11:57:37.877114   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl create minikube --distribution boot2docker --dst /Users/pierrebeaucamp/.minikube/machines/minikube --no-hdd
I0429 11:57:38.354808   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl set minikube --select-boot-device off --cpus 2 --memsize 4000 --videosize 64 --cpu-hotplug off --on-window-close keep-running --longer-battery-life on --3d-accelerate off --device-bootorder cdrom0
I0429 11:57:38.770433   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl set minikube --device-set cdrom0 --iface sata --position 0 --image /Users/pierrebeaucamp/.minikube/machines/minikube/boot2docker.iso --connect
I0429 11:57:39.053410   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl set minikube --device-add hdd --iface sata --position 1 --image /Users/pierrebeaucamp/.minikube/machines/minikube/disk.hdd --type plain --size 32
I0429 11:57:39.566670   54098 main.go:126] libmachine: Writing magic tar header
I0429 11:57:39.566843   54098 main.go:126] libmachine: Writing SSH key tar header
I0429 11:57:39.567332   54098 main.go:126] libmachine: HDS image path: /Users/pierrebeaucamp/.minikube/machines/minikube/disk.hdd/disk.hdd.0.{5fbaabe3-6958-40ff-92a7-860e329aab41}.hds
I0429 11:57:39.567549   54098 main.go:126] libmachine: executing: /usr/local/bin/prl_disk_tool convert --expanding --merge --hdd /Users/pierrebeaucamp/.minikube/machines/minikube/disk.hdd
I0429 11:57:39.711629   54098 main.go:126] libmachine: executing: /usr/local/bin/prl_disk_tool resize --hdd /Users/pierrebeaucamp/.minikube/machines/minikube/disk.hdd --size 20000
I0429 11:57:39.849220   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl set minikube --startup-view headless
I0429 11:57:40.140996   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl set minikube --shf-host-defined off
I0429 11:57:40.437934   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl set minikube --time-sync on
I0429 11:57:40.728246   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl set minikube --disable-timezone-sync on
I0429 11:57:41.005625   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl set minikube --shf-host on --shared-cloud off --shared-profile off --smart-mount off
I0429 11:57:41.298326   54098 main.go:126] libmachine: Starting Parallels Desktop VM...
I0429 11:57:41.298384   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl start minikube
I0429 11:57:42.234941   54098 main.go:126] libmachine: Waiting for VM to come online...
I0429 11:57:42.236696   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl list -i minikube
I0429 11:57:42.593906   54098 main.go:126] libmachine: Not there yet 1/60, error: IP lease not found for MAC address 001C42E8F8CB in: /Library/Preferences/Parallels/parallels_dhcp_leases

I0429 11:57:44.595135   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl list -i minikube
I0429 11:57:44.981360   54098 main.go:126] libmachine: Not there yet 2/60, error: IP lease not found for MAC address 001C42E8F8CB in: /Library/Preferences/Parallels/parallels_dhcp_leases

I0429 11:57:46.985276   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl list -i minikube
I0429 11:57:47.205567   54098 main.go:126] libmachine: Not there yet 3/60, error: IP lease not found for MAC address 001C42E8F8CB in: /Library/Preferences/Parallels/parallels_dhcp_leases

I0429 11:57:49.207228   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl list -i minikube
I0429 11:57:49.414354   54098 main.go:126] libmachine: Not there yet 4/60, error: IP lease not found for MAC address 001C42E8F8CB in: /Library/Preferences/Parallels/parallels_dhcp_leases

I0429 11:57:51.416388   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl list -i minikube
I0429 11:57:51.676954   54098 main.go:126] libmachine: Not there yet 5/60, error: IP lease not found for MAC address 001C42E8F8CB in: /Library/Preferences/Parallels/parallels_dhcp_leases

I0429 11:57:53.680747   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl list -i minikube
I0429 11:57:53.908046   54098 main.go:126] libmachine: Not there yet 6/60, error: IP lease not found for MAC address 001C42E8F8CB in: /Library/Preferences/Parallels/parallels_dhcp_leases

I0429 11:57:55.909069   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl list -i minikube
I0429 11:57:56.157238   54098 main.go:126] libmachine: Found lease: 10.211.55.10 for MAC: 001C42E8F8CB, expiring at 1619713674, leased for 1800 s.

I0429 11:57:56.157326   54098 main.go:126] libmachine: Found IP lease: 10.211.55.10 for MAC address 001C42E8F8CB

I0429 11:57:56.157357   54098 main.go:126] libmachine: Got an ip: 10.211.55.10
I0429 11:57:56.158218   54098 main.go:126] libmachine: executing: /usr/local/bin/prlsrvctl net info Shared
I0429 11:57:56.483186   54098 main.go:126] libmachine: IP address of Shared network adapter: 10.211.55.2
I0429 11:57:56.483484   54098 main.go:126] libmachine: All host interface addressess: [127.0.0.1/8 ::1/128 fe80::1/64 fe80::492:f46b:ac9b:e424/64 10.88.111.9/24 fe80::ec47:5bff:fe81:3ff4/64 fe80::ec47:5bff:fe81:3ff4/64 fe80::5875:413d:9a9b:f75c/64 fe80::da33:133e:8ef3:2a46/64 fe80::f521:f26e:f6c2:232d/64 fe80::d69:140d:5c05:8a11/64 10.211.55.2/24 fe80::61:f218:1203:29a4/64 10.37.129.2/24 fe80::1c8c:87e4:b408:632c/64]
I0429 11:57:56.483537   54098 main.go:126] libmachine: Parallels Shared network adapter is connected
I0429 11:57:56.483570   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl list minikube --output status --no-header
I0429 11:57:56.647962   54098 main.go:126] libmachine: Getting to WaitForSSH function...
I0429 11:57:56.648115   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl list minikube --output status --no-header
I0429 11:57:56.822046   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl list -i minikube
I0429 11:57:57.044024   54098 main.go:126] libmachine: Found lease: 10.211.55.10 for MAC: 001C42E8F8CB, expiring at 1619713674, leased for 1800 s.

I0429 11:57:57.044082   54098 main.go:126] libmachine: Found IP lease: 10.211.55.10 for MAC address 001C42E8F8CB

I0429 11:57:57.044185   54098 main.go:126] libmachine: Using SSH client type: native
I0429 11:57:57.044481   54098 main.go:126] libmachine: &{{{ 0 [] [] []} docker [0x4402320] 0x44022e0   [] 0s} 10.211.55.10 22  }
I0429 11:57:57.044509   54098 main.go:126] libmachine: About to run SSH command:
exit 0
I0429 11:57:57.111894   54098 main.go:126] libmachine: SSH cmd err, output: : 
I0429 11:57:57.112009   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl list minikube --output status --no-header
I0429 11:57:57.292144   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl list -i minikube
I0429 11:57:57.464955   54098 main.go:126] libmachine: Found lease: 10.211.55.10 for MAC: 001C42E8F8CB, expiring at 1619713674, leased for 1800 s.

I0429 11:57:57.465003   54098 main.go:126] libmachine: Found IP lease: 10.211.55.10 for MAC address 001C42E8F8CB

I0429 11:57:57.465026   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl list --info minikube
I0429 11:57:57.640966   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl list minikube --output status --no-header
I0429 11:57:57.803149   54098 machine.go:88] provisioning docker machine ...
I0429 11:57:57.803235   54098 buildroot.go:166] provisioning hostname "minikube"
I0429 11:57:57.803342   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl list minikube --output status --no-header
I0429 11:57:57.964479   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl list -i minikube
I0429 11:57:58.142727   54098 main.go:126] libmachine: Found lease: 10.211.55.10 for MAC: 001C42E8F8CB, expiring at 1619713674, leased for 1800 s.

I0429 11:57:58.142802   54098 main.go:126] libmachine: Found IP lease: 10.211.55.10 for MAC address 001C42E8F8CB

I0429 11:57:58.142890   54098 main.go:126] libmachine: Using SSH client type: native
I0429 11:57:58.143106   54098 main.go:126] libmachine: &{{{ 0 [] [] []} docker [0x4402320] 0x44022e0   [] 0s} 10.211.55.10 22  }
I0429 11:57:58.143135   54098 main.go:126] libmachine: About to run SSH command:
sudo hostname minikube && echo "minikube" | sudo tee /etc/hostname
I0429 11:57:58.226259   54098 main.go:126] libmachine: SSH cmd err, output: : minikube

I0429 11:57:58.226451   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl list minikube --output status --no-header
I0429 11:57:58.389706   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl list -i minikube
I0429 11:57:58.564685   54098 main.go:126] libmachine: Found lease: 10.211.55.10 for MAC: 001C42E8F8CB, expiring at 1619713674, leased for 1800 s.

I0429 11:57:58.564739   54098 main.go:126] libmachine: Found IP lease: 10.211.55.10 for MAC address 001C42E8F8CB

I0429 11:57:58.564849   54098 main.go:126] libmachine: Using SSH client type: native
I0429 11:57:58.565069   54098 main.go:126] libmachine: &{{{ 0 [] [] []} docker [0x4402320] 0x44022e0   [] 0s} 10.211.55.10 22  }
I0429 11:57:58.565101   54098 main.go:126] libmachine: About to run SSH command:

        if ! grep -xq '.*\sminikube' /etc/hosts; then
            if grep -xq '127.0.1.1\s.*' /etc/hosts; then
                sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 minikube/g' /etc/hosts;
            else 
                echo '127.0.1.1 minikube' | sudo tee -a /etc/hosts; 
            fi
        fi
I0429 11:57:58.631941   54098 main.go:126] libmachine: SSH cmd err, output: : 
I0429 11:57:58.632003   54098 buildroot.go:172] set auth options {CertDir:/Users/pierrebeaucamp/.minikube CaCertPath:/Users/pierrebeaucamp/.minikube/certs/ca.pem CaPrivateKeyPath:/Users/pierrebeaucamp/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/Users/pierrebeaucamp/.minikube/machines/server.pem ServerKeyPath:/Users/pierrebeaucamp/.minikube/machines/server-key.pem ClientKeyPath:/Users/pierrebeaucamp/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/Users/pierrebeaucamp/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/Users/pierrebeaucamp/.minikube}
I0429 11:57:58.632049   54098 buildroot.go:174] setting up certificates
I0429 11:57:58.632069   54098 provision.go:83] configureAuth start
I0429 11:57:58.632097   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl list minikube --output status --no-header
I0429 11:57:58.799512   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl list -i minikube
I0429 11:57:58.971143   54098 main.go:126] libmachine: Found lease: 10.211.55.10 for MAC: 001C42E8F8CB, expiring at 1619713674, leased for 1800 s.

I0429 11:57:58.971184   54098 main.go:126] libmachine: Found IP lease: 10.211.55.10 for MAC address 001C42E8F8CB

I0429 11:57:58.971209   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl list minikube --output status --no-header
I0429 11:57:59.138205   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl list -i minikube
I0429 11:57:59.314184   54098 main.go:126] libmachine: Found lease: 10.211.55.10 for MAC: 001C42E8F8CB, expiring at 1619713674, leased for 1800 s.

I0429 11:57:59.314246   54098 main.go:126] libmachine: Found IP lease: 10.211.55.10 for MAC address 001C42E8F8CB

I0429 11:57:59.314270   54098 provision.go:137] copyHostCerts
I0429 11:57:59.314311   54098 vm_assets.go:96] NewFileAsset: /Users/pierrebeaucamp/.minikube/certs/ca.pem -> /Users/pierrebeaucamp/.minikube/ca.pem
I0429 11:57:59.314429   54098 exec_runner.go:145] found /Users/pierrebeaucamp/.minikube/ca.pem, removing ...
I0429 11:57:59.314445   54098 exec_runner.go:190] rm: /Users/pierrebeaucamp/.minikube/ca.pem
I0429 11:57:59.314749   54098 exec_runner.go:152] cp: /Users/pierrebeaucamp/.minikube/certs/ca.pem --> /Users/pierrebeaucamp/.minikube/ca.pem (1099 bytes)
I0429 11:57:59.315072   54098 vm_assets.go:96] NewFileAsset: /Users/pierrebeaucamp/.minikube/certs/cert.pem -> /Users/pierrebeaucamp/.minikube/cert.pem
I0429 11:57:59.315146   54098 exec_runner.go:145] found /Users/pierrebeaucamp/.minikube/cert.pem, removing ...
I0429 11:57:59.315205   54098 exec_runner.go:190] rm: /Users/pierrebeaucamp/.minikube/cert.pem
I0429 11:57:59.315367   54098 exec_runner.go:152] cp: /Users/pierrebeaucamp/.minikube/certs/cert.pem --> /Users/pierrebeaucamp/.minikube/cert.pem (1143 bytes)
I0429 11:57:59.315613   54098 vm_assets.go:96] NewFileAsset: /Users/pierrebeaucamp/.minikube/certs/key.pem -> /Users/pierrebeaucamp/.minikube/key.pem
I0429 11:57:59.315686   54098 exec_runner.go:145] found /Users/pierrebeaucamp/.minikube/key.pem, removing ...
I0429 11:57:59.315702   54098 exec_runner.go:190] rm: /Users/pierrebeaucamp/.minikube/key.pem
I0429 11:57:59.315888   54098 exec_runner.go:152] cp: /Users/pierrebeaucamp/.minikube/certs/key.pem --> /Users/pierrebeaucamp/.minikube/key.pem (1675 bytes)
I0429 11:57:59.316568   54098 provision.go:111] generating server cert: /Users/pierrebeaucamp/.minikube/machines/server.pem ca-key=/Users/pierrebeaucamp/.minikube/certs/ca.pem private-key=/Users/pierrebeaucamp/.minikube/certs/ca-key.pem org=pierrebeaucamp.minikube san=[10.211.55.10 10.211.55.10 localhost 127.0.0.1 minikube minikube]
I0429 11:57:59.454892   54098 provision.go:165] copyRemoteCerts
I0429 11:57:59.455033   54098 ssh_runner.go:149] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0429 11:57:59.455074   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl list minikube --output status --no-header
I0429 11:57:59.621300   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl list -i minikube
I0429 11:57:59.794965   54098 main.go:126] libmachine: Found lease: 10.211.55.10 for MAC: 001C42E8F8CB, expiring at 1619713674, leased for 1800 s.

I0429 11:57:59.795005   54098 main.go:126] libmachine: Found IP lease: 10.211.55.10 for MAC address 001C42E8F8CB

I0429 11:57:59.795026   54098 sshutil.go:53] new ssh client: &{IP:10.211.55.10 Port:22 SSHKeyPath:/Users/pierrebeaucamp/.minikube/machines/minikube/id_rsa Username:docker}
I0429 11:57:59.835787   54098 vm_assets.go:96] NewFileAsset: /Users/pierrebeaucamp/.minikube/machines/server-key.pem -> /etc/docker/server-key.pem
I0429 11:57:59.835923   54098 ssh_runner.go:316] scp /Users/pierrebeaucamp/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1675 bytes)
I0429 11:57:59.851072   54098 vm_assets.go:96] NewFileAsset: /Users/pierrebeaucamp/.minikube/certs/ca.pem -> /etc/docker/ca.pem
I0429 11:57:59.851204   54098 ssh_runner.go:316] scp /Users/pierrebeaucamp/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1099 bytes)
I0429 11:57:59.863994   54098 vm_assets.go:96] NewFileAsset: /Users/pierrebeaucamp/.minikube/machines/server.pem -> /etc/docker/server.pem
I0429 11:57:59.864126   54098 ssh_runner.go:316] scp /Users/pierrebeaucamp/.minikube/machines/server.pem --> /etc/docker/server.pem (1224 bytes)
I0429 11:57:59.880858   54098 provision.go:86] duration metric: configureAuth took 1.248838917s
I0429 11:57:59.880908   54098 buildroot.go:189] setting minikube options for container-runtime
I0429 11:57:59.881192   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl list minikube --output status --no-header
I0429 11:58:00.051836   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl list -i minikube
I0429 11:58:00.240437   54098 main.go:126] libmachine: Found lease: 10.211.55.10 for MAC: 001C42E8F8CB, expiring at 1619713674, leased for 1800 s.

I0429 11:58:00.240483   54098 main.go:126] libmachine: Found IP lease: 10.211.55.10 for MAC address 001C42E8F8CB

I0429 11:58:00.240559   54098 main.go:126] libmachine: Using SSH client type: native
I0429 11:58:00.240760   54098 main.go:126] libmachine: &{{{ 0 [] [] []} docker [0x4402320] 0x44022e0   [] 0s} 10.211.55.10 22  }
I0429 11:58:00.240815   54098 main.go:126] libmachine: About to run SSH command:
df --output=fstype / | tail -n 1
I0429 11:58:00.312636   54098 main.go:126] libmachine: SSH cmd err, output: : tmpfs

I0429 11:58:00.312726   54098 buildroot.go:70] root file system type: tmpfs
I0429 11:58:00.313074   54098 provision.go:296] Updating docker unit: /lib/systemd/system/docker.service ...
I0429 11:58:00.313144   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl list minikube --output status --no-header
I0429 11:58:00.489310   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl list -i minikube
I0429 11:58:00.675848   54098 main.go:126] libmachine: Found lease: 10.211.55.10 for MAC: 001C42E8F8CB, expiring at 1619713674, leased for 1800 s.

I0429 11:58:00.675903   54098 main.go:126] libmachine: Found IP lease: 10.211.55.10 for MAC address 001C42E8F8CB

I0429 11:58:00.675990   54098 main.go:126] libmachine: Using SSH client type: native
I0429 11:58:00.676289   54098 main.go:126] libmachine: &{{{ 0 [] [] []} docker [0x4402320] 0x44022e0   [] 0s} 10.211.55.10 22  }
I0429 11:58:00.676413   54098 main.go:126] libmachine: About to run SSH command:
sudo mkdir -p /lib/systemd/system && printf %s "[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network.target  minikube-automount.service docker.socket
Requires= minikube-automount.service docker.socket 
StartLimitBurst=3
StartLimitIntervalSec=60

[Service]
Type=notify
Restart=on-failure

# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
#  Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.

# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=parallels --insecure-registry 10.96.0.0/12 
ExecReload=/bin/kill -s HUP \$MAINPID

# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity

# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0

# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes

# kill only the docker process, not all processes in the cgroup
KillMode=process

[Install]
WantedBy=multi-user.target
" | sudo tee /lib/systemd/system/docker.service.new
I0429 11:58:00.751744   54098 main.go:126] libmachine: SSH cmd err, output: : [Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network.target  minikube-automount.service docker.socket
Requires= minikube-automount.service docker.socket 
StartLimitBurst=3
StartLimitIntervalSec=60

[Service]
Type=notify
Restart=on-failure

# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
#  Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.

# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=parallels --insecure-registry 10.96.0.0/12 
ExecReload=/bin/kill -s HUP $MAINPID

# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity

# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0

# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes

# kill only the docker process, not all processes in the cgroup
KillMode=process

[Install]
WantedBy=multi-user.target

I0429 11:58:00.751878   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl list minikube --output status --no-header
I0429 11:58:00.934111   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl list -i minikube
I0429 11:58:01.163161   54098 main.go:126] libmachine: Found lease: 10.211.55.10 for MAC: 001C42E8F8CB, expiring at 1619713674, leased for 1800 s.

I0429 11:58:01.163220   54098 main.go:126] libmachine: Found IP lease: 10.211.55.10 for MAC address 001C42E8F8CB

I0429 11:58:01.163296   54098 main.go:126] libmachine: Using SSH client type: native
I0429 11:58:01.163553   54098 main.go:126] libmachine: &{{{ 0 [] [] []} docker [0x4402320] 0x44022e0   [] 0s} 10.211.55.10 22  }
I0429 11:58:01.163589   54098 main.go:126] libmachine: About to run SSH command:
sudo diff -u /lib/systemd/system/docker.service /lib/systemd/system/docker.service.new || { sudo mv /lib/systemd/system/docker.service.new /lib/systemd/system/docker.service; sudo systemctl -f daemon-reload && sudo systemctl -f enable docker && sudo systemctl -f restart docker; }
I0429 11:58:01.908209   54098 main.go:126] libmachine: SSH cmd err, output: : diff: can't stat '/lib/systemd/system/docker.service': No such file or directory
Created symlink /etc/systemd/system/multi-user.target.wants/docker.service → /usr/lib/systemd/system/docker.service.

I0429 11:58:01.908287   54098 machine.go:91] provisioned docker machine in 4.105317781s
I0429 11:58:01.908337   54098 client.go:171] LocalClient.Create took 25.077494274s
I0429 11:58:01.908406   54098 start.go:168] duration metric: libmachine.API.Create for "minikube" took 25.077851616s
I0429 11:58:01.908452   54098 start.go:267] post-start starting for "minikube" (driver="parallels")
I0429 11:58:01.908480   54098 start.go:277] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0429 11:58:01.908601   54098 ssh_runner.go:149] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0429 11:58:01.908628   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl list minikube --output status --no-header
I0429 11:58:02.087734   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl list -i minikube
I0429 11:58:02.274951   54098 main.go:126] libmachine: Found lease: 10.211.55.10 for MAC: 001C42E8F8CB, expiring at 1619713674, leased for 1800 s.

I0429 11:58:02.274989   54098 main.go:126] libmachine: Found IP lease: 10.211.55.10 for MAC address 001C42E8F8CB

I0429 11:58:02.275003   54098 sshutil.go:53] new ssh client: &{IP:10.211.55.10 Port:22 SSHKeyPath:/Users/pierrebeaucamp/.minikube/machines/minikube/id_rsa Username:docker}
I0429 11:58:02.314083   54098 ssh_runner.go:149] Run: cat /etc/os-release
I0429 11:58:02.318434   54098 info.go:137] Remote host: Buildroot 2020.02.10
I0429 11:58:02.318478   54098 filesync.go:118] Scanning /Users/pierrebeaucamp/.minikube/addons for local assets ...
I0429 11:58:02.318757   54098 filesync.go:118] Scanning /Users/pierrebeaucamp/.minikube/files for local assets ...
I0429 11:58:02.318874   54098 start.go:270] post-start completed in 410.411263ms
I0429 11:58:02.319507   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl list minikube --output status --no-header
I0429 11:58:02.483298   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl list -i minikube
I0429 11:58:02.666322   54098 main.go:126] libmachine: Found lease: 10.211.55.10 for MAC: 001C42E8F8CB, expiring at 1619713674, leased for 1800 s.

I0429 11:58:02.666363   54098 main.go:126] libmachine: Found IP lease: 10.211.55.10 for MAC address 001C42E8F8CB

I0429 11:58:02.666509   54098 profile.go:148] Saving config to /Users/pierrebeaucamp/.minikube/profiles/minikube/config.json ...
I0429 11:58:02.666976   54098 start.go:129] duration metric: createHost completed in 25.841356428s
I0429 11:58:02.666994   54098 start.go:80] releasing machines lock for "minikube", held for 25.84149312s
I0429 11:58:02.667033   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl list minikube --output status --no-header
I0429 11:58:02.831040   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl list -i minikube
I0429 11:58:03.012753   54098 main.go:126] libmachine: Found lease: 10.211.55.10 for MAC: 001C42E8F8CB, expiring at 1619713674, leased for 1800 s.

I0429 11:58:03.012802   54098 main.go:126] libmachine: Found IP lease: 10.211.55.10 for MAC address 001C42E8F8CB

I0429 11:58:03.013773   54098 ssh_runner.go:149] Run: systemctl --version
I0429 11:58:03.013821   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl list minikube --output status --no-header
I0429 11:58:03.013869   54098 ssh_runner.go:149] Run: curl -sS -m 2 https://k8s.gcr.io/
I0429 11:58:03.013948   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl list minikube --output status --no-header
I0429 11:58:03.226040   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl list -i minikube
I0429 11:58:03.230760   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl list -i minikube
I0429 11:58:03.444277   54098 main.go:126] libmachine: Found lease: 10.211.55.10 for MAC: 001C42E8F8CB, expiring at 1619713674, leased for 1800 s.

I0429 11:58:03.444331   54098 main.go:126] libmachine: Found IP lease: 10.211.55.10 for MAC address 001C42E8F8CB

I0429 11:58:03.444371   54098 sshutil.go:53] new ssh client: &{IP:10.211.55.10 Port:22 SSHKeyPath:/Users/pierrebeaucamp/.minikube/machines/minikube/id_rsa Username:docker}
I0429 11:58:03.456843   54098 main.go:126] libmachine: Found lease: 10.211.55.10 for MAC: 001C42E8F8CB, expiring at 1619713674, leased for 1800 s.

I0429 11:58:03.456891   54098 main.go:126] libmachine: Found IP lease: 10.211.55.10 for MAC address 001C42E8F8CB

I0429 11:58:03.456918   54098 sshutil.go:53] new ssh client: &{IP:10.211.55.10 Port:22 SSHKeyPath:/Users/pierrebeaucamp/.minikube/machines/minikube/id_rsa Username:docker}
I0429 11:58:03.493309   54098 preload.go:97] Checking if preload exists for k8s version v1.20.2 and runtime docker
I0429 11:58:03.493630   54098 preload.go:105] Found local preload: /Users/pierrebeaucamp/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v10-v1.20.2-docker-overlay2-amd64.tar.lz4
I0429 11:58:03.494080   54098 ssh_runner.go:149] Run: docker images --format {{.Repository}}:{{.Tag}}
I0429 11:58:03.525875   54098 docker.go:455] Got preloaded images: 
I0429 11:58:03.525922   54098 docker.go:461] k8s.gcr.io/kube-proxy:v1.20.2 wasn't preloaded
I0429 11:58:03.526007   54098 ssh_runner.go:149] Run: sudo cat /var/lib/docker/image/overlay2/repositories.json
I0429 11:58:03.636037   54098 ssh_runner.go:149] Run: which lz4
I0429 11:58:03.658527   54098 vm_assets.go:96] NewFileAsset: /Users/pierrebeaucamp/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v10-v1.20.2-docker-overlay2-amd64.tar.lz4 -> /preloaded.tar.lz4
I0429 11:58:03.658895   54098 ssh_runner.go:149] Run: stat -c "%s %y" /preloaded.tar.lz4
I0429 11:58:03.666594   54098 ssh_runner.go:306] existence check for /preloaded.tar.lz4: stat -c "%s %y" /preloaded.tar.lz4: Process exited with status 1
stdout:

stderr:
stat: cannot stat '/preloaded.tar.lz4': No such file or directory
I0429 11:58:03.666711   54098 ssh_runner.go:316] scp /Users/pierrebeaucamp/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v10-v1.20.2-docker-overlay2-amd64.tar.lz4 --> /preloaded.tar.lz4 (515595048 bytes)
I0429 11:58:07.208297   54098 docker.go:420] Took 3.549749 seconds to copy over tarball
I0429 11:58:07.208444   54098 ssh_runner.go:149] Run: sudo tar -I lz4 -C /var -xf /preloaded.tar.lz4
I0429 11:58:15.909298   54098 ssh_runner.go:189] Completed: sudo tar -I lz4 -C /var -xf /preloaded.tar.lz4: (8.701314125s)
I0429 11:58:15.909823   54098 ssh_runner.go:100] rm: /preloaded.tar.lz4
I0429 11:58:15.942426   54098 ssh_runner.go:149] Run: sudo cat /var/lib/docker/image/overlay2/repositories.json
I0429 11:58:15.950242   54098 ssh_runner.go:316] scp memory --> /var/lib/docker/image/overlay2/repositories.json (3125 bytes)
I0429 11:58:15.962491   54098 ssh_runner.go:149] Run: sudo systemctl daemon-reload
I0429 11:58:16.090421   54098 ssh_runner.go:149] Run: sudo systemctl restart docker
I0429 11:58:18.067897   54098 ssh_runner.go:189] Completed: sudo systemctl restart docker: (1.977502846s)
I0429 11:58:18.068072   54098 ssh_runner.go:149] Run: sudo systemctl is-active --quiet service containerd
I0429 11:58:18.079126   54098 ssh_runner.go:149] Run: sudo systemctl cat docker.service
I0429 11:58:18.091959   54098 ssh_runner.go:149] Run: sudo systemctl is-active --quiet service containerd
I0429 11:58:18.103864   54098 ssh_runner.go:149] Run: sudo systemctl is-active --quiet service crio
I0429 11:58:18.114898   54098 ssh_runner.go:149] Run: sudo systemctl stop -f crio
I0429 11:58:18.146350   54098 ssh_runner.go:149] Run: sudo systemctl is-active --quiet service crio
I0429 11:58:18.156847   54098 ssh_runner.go:149] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///var/run/dockershim.sock
image-endpoint: unix:///var/run/dockershim.sock
" | sudo tee /etc/crictl.yaml"
I0429 11:58:18.169060   54098 ssh_runner.go:149] Run: sudo systemctl daemon-reload
I0429 11:58:18.290634   54098 ssh_runner.go:149] Run: sudo systemctl start docker
I0429 11:58:18.300471   54098 ssh_runner.go:149] Run: docker version --format {{.Server.Version}}
I0429 11:58:18.392239   54098 out.go:184] * Preparing Kubernetes v1.20.2 on Docker 20.10.4 ...
* Preparing Kubernetes v1.20.2 on Docker 20.10.4 ...
I0429 11:58:18.839632   54098 ssh_runner.go:149] Run: grep 10.211.55.2  host.minikube.internal$ /etc/hosts
I0429 11:58:18.850587   54098 ssh_runner.go:149] Run: /bin/bash -c "{ grep -v '\thost.minikube.internal$' /etc/hosts; echo "10.211.55.2 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ /etc/hosts"
I0429 11:58:18.866275   54098 preload.go:97] Checking if preload exists for k8s version v1.20.2 and runtime docker
I0429 11:58:18.869365   54098 preload.go:105] Found local preload: /Users/pierrebeaucamp/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v10-v1.20.2-docker-overlay2-amd64.tar.lz4
I0429 11:58:18.870274   54098 ssh_runner.go:149] Run: docker images --format {{.Repository}}:{{.Tag}}
I0429 11:58:18.927304   54098 docker.go:455] Got preloaded images: -- stdout --
gcr.io/k8s-minikube/storage-provisioner:v5
k8s.gcr.io/kube-proxy:v1.20.2
k8s.gcr.io/kube-controller-manager:v1.20.2
k8s.gcr.io/kube-apiserver:v1.20.2
k8s.gcr.io/kube-scheduler:v1.20.2
kubernetesui/dashboard:v2.1.0
k8s.gcr.io/etcd:3.4.13-0
k8s.gcr.io/coredns:1.7.0
kubernetesui/metrics-scraper:v1.0.4
k8s.gcr.io/pause:3.2

-- /stdout --
I0429 11:58:18.927489   54098 docker.go:392] Images already preloaded, skipping extraction
I0429 11:58:18.927722   54098 ssh_runner.go:149] Run: docker images --format {{.Repository}}:{{.Tag}}
I0429 11:58:18.988583   54098 docker.go:455] Got preloaded images: -- stdout --
gcr.io/k8s-minikube/storage-provisioner:v5
k8s.gcr.io/kube-proxy:v1.20.2
k8s.gcr.io/kube-apiserver:v1.20.2
k8s.gcr.io/kube-controller-manager:v1.20.2
k8s.gcr.io/kube-scheduler:v1.20.2
kubernetesui/dashboard:v2.1.0
k8s.gcr.io/etcd:3.4.13-0
k8s.gcr.io/coredns:1.7.0
kubernetesui/metrics-scraper:v1.0.4
k8s.gcr.io/pause:3.2

-- /stdout --
I0429 11:58:18.988665   54098 cache_images.go:74] Images are preloaded, skipping loading
I0429 11:58:18.989910   54098 ssh_runner.go:149] Run: docker info --format {{.CgroupDriver}}
I0429 11:58:19.052513   54098 cni.go:81] Creating CNI manager for ""
I0429 11:58:19.052562   54098 cni.go:153] CNI unnecessary in this configuration, recommending no CNI
I0429 11:58:19.052593   54098 kubeadm.go:87] Using pod CIDR: 10.244.0.0/16
I0429 11:58:19.052618   54098 kubeadm.go:153] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:10.211.55.10 APIServerPort:8443 KubernetesVersion:v1.20.2 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:minikube NodeName:minikube DNSDomain:cluster.local CRISocket:/var/run/dockershim.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "10.211.55.10"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NoTaintMaster:true NodeIP:10.211.55.10 CgroupDriver:systemd ClientCAFile:/var/lib/minikube/certs/ca.crt StaticPodPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[]}
I0429 11:58:19.052741   54098 kubeadm.go:157] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta2
kind: InitConfiguration
localAPIEndpoint:
  advertiseAddress: 10.211.55.10
  bindPort: 8443
bootstrapTokens:
  - groups:
      - system:bootstrappers:kubeadm:default-node-token
    ttl: 24h0m0s
    usages:
      - signing
      - authentication
nodeRegistration:
  criSocket: /var/run/dockershim.sock
  name: "minikube"
  kubeletExtraArgs:
    node-ip: 10.211.55.10
  taints: []
---
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
apiServer:
  certSANs: ["127.0.0.1", "localhost", "10.211.55.10"]
  extraArgs:
    enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
  extraArgs:
    allocate-node-cidrs: "true"
    leader-elect: "false"
scheduler:
  extraArgs:
    leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
dns:
  type: CoreDNS
etcd:
  local:
    dataDir: /var/lib/minikube/etcd
    extraArgs:
      proxy-refresh-interval: "70000"
kubernetesVersion: v1.20.2
networking:
  dnsDomain: cluster.local
  podSubnet: "10.244.0.0/16"
  serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
  x509:
    clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: systemd
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
  nodefs.available: "0%"
  nodefs.inodesFree: "0%"
  imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249

I0429 11:58:19.052853   54098 kubeadm.go:897] kubelet [Unit]
Wants=docker.socket

[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.20.2/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime=docker --hostname-override=minikube --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=10.211.55.10

[Install]
 config:
{KubernetesVersion:v1.20.2 ClusterName:minikube Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:}
I0429 11:58:19.052946   54098 ssh_runner.go:149] Run: sudo ls /var/lib/minikube/binaries/v1.20.2
I0429 11:58:19.059991   54098 binaries.go:44] Found k8s binaries, skipping transfer
I0429 11:58:19.060114   54098 ssh_runner.go:149] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0429 11:58:19.067510   54098 ssh_runner.go:316] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (334 bytes)
I0429 11:58:19.080403   54098 ssh_runner.go:316] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0429 11:58:19.095166   54098 ssh_runner.go:316] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (1839 bytes)
I0429 11:58:19.108687   54098 ssh_runner.go:149] Run: grep 10.211.55.10 control-plane.minikube.internal$ /etc/hosts
I0429 11:58:19.115854   54098 ssh_runner.go:149] Run: /bin/bash -c "{ grep -v '\tcontrol-plane.minikube.internal$' /etc/hosts; echo "10.211.55.10   control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ /etc/hosts"
I0429 11:58:19.125603   54098 certs.go:52] Setting up /Users/pierrebeaucamp/.minikube/profiles/minikube for IP: 10.211.55.10
I0429 11:58:19.125779   54098 certs.go:171] skipping minikubeCA CA generation: /Users/pierrebeaucamp/.minikube/ca.key
I0429 11:58:19.125885   54098 certs.go:171] skipping proxyClientCA CA generation: /Users/pierrebeaucamp/.minikube/proxy-client-ca.key
I0429 11:58:19.126030   54098 certs.go:286] generating minikube-user signed cert: /Users/pierrebeaucamp/.minikube/profiles/minikube/client.key
I0429 11:58:19.126051   54098 crypto.go:69] Generating cert /Users/pierrebeaucamp/.minikube/profiles/minikube/client.crt with IP's: []
I0429 11:58:19.343134   54098 crypto.go:157] Writing cert to /Users/pierrebeaucamp/.minikube/profiles/minikube/client.crt ...
I0429 11:58:19.343174   54098 lock.go:36] WriteFile acquiring /Users/pierrebeaucamp/.minikube/profiles/minikube/client.crt: {Name:mk4f129ead6b3036ebbc50cb25398c9218af87b6 Clock:{} Delay:500ms Timeout:1m0s Cancel:}
I0429 11:58:19.343542   54098 crypto.go:165] Writing key to /Users/pierrebeaucamp/.minikube/profiles/minikube/client.key ...
I0429 11:58:19.343566   54098 lock.go:36] WriteFile acquiring /Users/pierrebeaucamp/.minikube/profiles/minikube/client.key: {Name:mk3f5a0f96860100d6dafd9d5b1662266f8b0eee Clock:{} Delay:500ms Timeout:1m0s Cancel:}
I0429 11:58:19.343801   54098 certs.go:286] generating minikube signed cert: /Users/pierrebeaucamp/.minikube/profiles/minikube/apiserver.key.0a950026
I0429 11:58:19.343810   54098 crypto.go:69] Generating cert /Users/pierrebeaucamp/.minikube/profiles/minikube/apiserver.crt.0a950026 with IP's: [10.211.55.10 10.96.0.1 127.0.0.1 10.0.0.1 10.211.55.4]
I0429 11:58:19.467966   54098 crypto.go:157] Writing cert to /Users/pierrebeaucamp/.minikube/profiles/minikube/apiserver.crt.0a950026 ...
I0429 11:58:19.468006   54098 lock.go:36] WriteFile acquiring /Users/pierrebeaucamp/.minikube/profiles/minikube/apiserver.crt.0a950026: {Name:mk211e0f207097ac8bdaa6b1c05a067847b76da8 Clock:{} Delay:500ms Timeout:1m0s Cancel:}
I0429 11:58:19.468552   54098 crypto.go:165] Writing key to /Users/pierrebeaucamp/.minikube/profiles/minikube/apiserver.key.0a950026 ...
I0429 11:58:19.468569   54098 lock.go:36] WriteFile acquiring /Users/pierrebeaucamp/.minikube/profiles/minikube/apiserver.key.0a950026: {Name:mk1eeecd7cb36429cefa6aed5c3a1fb1dc86241e Clock:{} Delay:500ms Timeout:1m0s Cancel:}
I0429 11:58:19.469068   54098 certs.go:297] copying /Users/pierrebeaucamp/.minikube/profiles/minikube/apiserver.crt.0a950026 -> /Users/pierrebeaucamp/.minikube/profiles/minikube/apiserver.crt
I0429 11:58:19.470072   54098 certs.go:301] copying /Users/pierrebeaucamp/.minikube/profiles/minikube/apiserver.key.0a950026 -> /Users/pierrebeaucamp/.minikube/profiles/minikube/apiserver.key
I0429 11:58:19.470425   54098 certs.go:286] generating aggregator signed cert: /Users/pierrebeaucamp/.minikube/profiles/minikube/proxy-client.key
I0429 11:58:19.470436   54098 crypto.go:69] Generating cert /Users/pierrebeaucamp/.minikube/profiles/minikube/proxy-client.crt with IP's: []
I0429 11:58:19.528311   54098 crypto.go:157] Writing cert to /Users/pierrebeaucamp/.minikube/profiles/minikube/proxy-client.crt ...
I0429 11:58:19.528338   54098 lock.go:36] WriteFile acquiring /Users/pierrebeaucamp/.minikube/profiles/minikube/proxy-client.crt: {Name:mk70527a687fb3db6411fafe93d356917cafd22a Clock:{} Delay:500ms Timeout:1m0s Cancel:}
I0429 11:58:19.528634   54098 crypto.go:165] Writing key to /Users/pierrebeaucamp/.minikube/profiles/minikube/proxy-client.key ...
I0429 11:58:19.528645   54098 lock.go:36] WriteFile acquiring /Users/pierrebeaucamp/.minikube/profiles/minikube/proxy-client.key: {Name:mk7f391570a43ef8c807adeface22eb86d72ae9b Clock:{} Delay:500ms Timeout:1m0s Cancel:}
I0429 11:58:19.528863   54098 vm_assets.go:96] NewFileAsset: /Users/pierrebeaucamp/.minikube/profiles/minikube/apiserver.crt -> /var/lib/minikube/certs/apiserver.crt
I0429 11:58:19.528897   54098 vm_assets.go:96] NewFileAsset: /Users/pierrebeaucamp/.minikube/profiles/minikube/apiserver.key -> /var/lib/minikube/certs/apiserver.key
I0429 11:58:19.528924   54098 vm_assets.go:96] NewFileAsset: /Users/pierrebeaucamp/.minikube/profiles/minikube/proxy-client.crt -> /var/lib/minikube/certs/proxy-client.crt
I0429 11:58:19.528970   54098 vm_assets.go:96] NewFileAsset: /Users/pierrebeaucamp/.minikube/profiles/minikube/proxy-client.key -> /var/lib/minikube/certs/proxy-client.key
I0429 11:58:19.528997   54098 vm_assets.go:96] NewFileAsset: /Users/pierrebeaucamp/.minikube/ca.crt -> /var/lib/minikube/certs/ca.crt
I0429 11:58:19.529035   54098 vm_assets.go:96] NewFileAsset: /Users/pierrebeaucamp/.minikube/ca.key -> /var/lib/minikube/certs/ca.key
I0429 11:58:19.529059   54098 vm_assets.go:96] NewFileAsset: /Users/pierrebeaucamp/.minikube/proxy-client-ca.crt -> /var/lib/minikube/certs/proxy-client-ca.crt
I0429 11:58:19.529082   54098 vm_assets.go:96] NewFileAsset: /Users/pierrebeaucamp/.minikube/proxy-client-ca.key -> /var/lib/minikube/certs/proxy-client-ca.key
I0429 11:58:19.529214   54098 certs.go:361] found cert: /Users/pierrebeaucamp/.minikube/certs/Users/pierrebeaucamp/.minikube/certs/ca-key.pem (1675 bytes)
I0429 11:58:19.529667   54098 certs.go:361] found cert: /Users/pierrebeaucamp/.minikube/certs/Users/pierrebeaucamp/.minikube/certs/ca.pem (1099 bytes)
I0429 11:58:19.530218   54098 certs.go:361] found cert: /Users/pierrebeaucamp/.minikube/certs/Users/pierrebeaucamp/.minikube/certs/cert.pem (1143 bytes)
I0429 11:58:19.530731   54098 certs.go:361] found cert: /Users/pierrebeaucamp/.minikube/certs/Users/pierrebeaucamp/.minikube/certs/key.pem (1675 bytes)
I0429 11:58:19.531323   54098 vm_assets.go:96] NewFileAsset: /Users/pierrebeaucamp/.minikube/ca.crt -> /usr/share/ca-certificates/minikubeCA.pem
I0429 11:58:19.535473   54098 ssh_runner.go:316] scp /Users/pierrebeaucamp/.minikube/profiles/minikube/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1407 bytes)
I0429 11:58:19.554743   54098 ssh_runner.go:316] scp /Users/pierrebeaucamp/.minikube/profiles/minikube/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1679 bytes)
I0429 11:58:19.569059   54098 ssh_runner.go:316] scp /Users/pierrebeaucamp/.minikube/profiles/minikube/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0429 11:58:19.582307   54098 ssh_runner.go:316] scp /Users/pierrebeaucamp/.minikube/profiles/minikube/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1679 bytes)
I0429 11:58:19.595751   54098 ssh_runner.go:316] scp /Users/pierrebeaucamp/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0429 11:58:19.609764   54098 ssh_runner.go:316] scp /Users/pierrebeaucamp/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1679 bytes)
I0429 11:58:19.623470   54098 ssh_runner.go:316] scp /Users/pierrebeaucamp/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0429 11:58:19.637605   54098 ssh_runner.go:316] scp /Users/pierrebeaucamp/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1675 bytes)
I0429 11:58:19.650847   54098 ssh_runner.go:316] scp /Users/pierrebeaucamp/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0429 11:58:19.663531   54098 ssh_runner.go:316] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0429 11:58:19.673570   54098 ssh_runner.go:149] Run: openssl version
I0429 11:58:19.679119   54098 ssh_runner.go:149] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0429 11:58:19.685219   54098 ssh_runner.go:149] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0429 11:58:19.688901   54098 certs.go:402] hashing: -rw-r--r-- 1 root root 1111 Apr 27 20:18 /usr/share/ca-certificates/minikubeCA.pem
I0429 11:58:19.688999   54098 ssh_runner.go:149] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0429 11:58:19.694978   54098 ssh_runner.go:149] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0429 11:58:19.701555   54098 kubeadm.go:386] StartCluster: {Name:minikube KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube/iso/minikube-v1.19.0.iso KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.20@sha256:0250dab3644403384bd54f566921c6b57138eecffbb861f9392feef9b2ec44f6 Memory:4000 CPUs:2 DiskSize:20000 VMDriver: Driver:parallels HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.99.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.2 ClusterName:minikube Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP:10.211.55.10 Port:8443 KubernetesVersion:v1.20.2 ControlPlane:true Worker:true}] Addons:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop: ExposedPorts:[] ListenAddress: Network: MultiNodeRequested:false}
I0429 11:58:19.701725   54098 ssh_runner.go:149] Run: docker ps --filter status=paused --filter=name=k8s_.*_(kube-system)_ --format={{.ID}}
I0429 11:58:19.729065   54098 ssh_runner.go:149] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I0429 11:58:19.735474   54098 ssh_runner.go:149] Run: sudo cp /var/tmp/minikube/kubeadm.yaml.new /var/tmp/minikube/kubeadm.yaml
I0429 11:58:19.741871   54098 ssh_runner.go:149] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0429 11:58:19.746963   54098 kubeadm.go:151] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:

stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0429 11:58:19.747038   54098 ssh_runner.go:240] Start: /bin/bash -c "sudo env PATH=/var/lib/minikube/binaries/v1.20.2:$PATH kubeadm init --config /var/tmp/minikube/kubeadm.yaml  --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,Mem"
I0429 11:58:20.275579   54098 out.go:184]   - Generating certificates and keys ...
  - Generating certificates and keys ...
I0429 11:58:22.468228   54098 out.go:184]   - Booting up control plane ...
  - Booting up control plane ...
I0429 11:58:34.085900   54098 out.go:184]   - Configuring RBAC rules ...
  - Configuring RBAC rules ...
I0429 11:58:34.834204   54098 cni.go:81] Creating CNI manager for ""
I0429 11:58:34.834278   54098 cni.go:153] CNI unnecessary in this configuration, recommending no CNI
I0429 11:58:34.834323   54098 ssh_runner.go:149] Run: /bin/bash -c "cat /proc/$(pgrep kube-apiserver)/oom_adj"
I0429 11:58:34.834527   54098 ssh_runner.go:149] Run: sudo /var/lib/minikube/binaries/v1.20.2/kubectl create clusterrolebinding minikube-rbac --clusterrole=cluster-admin --serviceaccount=kube-system:default --kubeconfig=/var/lib/minikube/kubeconfig
I0429 11:58:34.834657   54098 ssh_runner.go:149] Run: sudo /var/lib/minikube/binaries/v1.20.2/kubectl label nodes minikube.k8s.io/version=v1.19.0 minikube.k8s.io/commit=15cede53bdc5fe242228853e737333b09d4336b5 minikube.k8s.io/name=minikube minikube.k8s.io/updated_at=2021_04_29T11_58_34_0700 --all --overwrite --kubeconfig=/var/lib/minikube/kubeconfig
I0429 11:58:35.317136   54098 ops.go:34] apiserver oom_adj: -16
I0429 11:58:35.317258   54098 kubeadm.go:973] duration metric: took 482.871436ms to wait for elevateKubeSystemPrivileges.
I0429 11:58:35.318529   54098 kubeadm.go:388] StartCluster complete in 15.617869804s
I0429 11:58:35.318602   54098 settings.go:142] acquiring lock: {Name:mka2772ff9d133b4dc23d372d749124ed3832ac4 Clock:{} Delay:500ms Timeout:1m0s Cancel:}
I0429 11:58:35.318777   54098 settings.go:150] Updating kubeconfig:  /Users/pierrebeaucamp/.kube/config
I0429 11:58:35.323861   54098 lock.go:36] WriteFile acquiring /Users/pierrebeaucamp/.kube/config: {Name:mk327349d16ac5e7f9ba15222ae29152c80fe972 Clock:{} Delay:500ms Timeout:1m0s Cancel:}
I0429 11:58:35.326473   54098 loader.go:379] Config loaded from file:  /Users/pierrebeaucamp/.kube/config
I0429 11:58:35.334754   54098 kapi.go:59] client config for minikube: &rest.Config{Host:"https://10.211.55.10:8443", APIPath:"", ContentConfig:rest.ContentConfig{AcceptContentTypes:"", ContentType:"", GroupVersion:(*schema.GroupVersion)(nil), NegotiatedSerializer:runtime.NegotiatedSerializer(nil)}, Username:"", Password:"", BearerToken:"", BearerTokenFile:"", Impersonate:rest.ImpersonationConfig{UserName:"", Groups:[]string(nil), Extra:map[string][]string(nil)}, AuthProvider:, AuthConfigPersister:rest.AuthProviderConfigPersister(nil), ExecProvider:, TLSClientConfig:rest.sanitizedTLSClientConfig{Insecure:false, ServerName:"", CertFile:"/Users/pierrebeaucamp/.minikube/profiles/minikube/client.crt", KeyFile:"/Users/pierrebeaucamp/.minikube/profiles/minikube/client.key", CAFile:"/Users/pierrebeaucamp/.minikube/ca.crt", CertData:[]uint8(nil), KeyData:[]uint8(nil), CAData:[]uint8(nil), NextProtos:[]string(nil)}, UserAgent:"", DisableCompression:false, Transport:http.RoundTripper(nil), WrapTransport:(transport.WrapperFunc)(0x567fdc0), QPS:0, Burst:0, RateLimiter:flowcontrol.RateLimiter(nil), WarningHandler:rest.WarningHandler(nil), Timeout:0, Dial:(func(context.Context, string, string) (net.Conn, error))(nil), Proxy:(func(*http.Request) (*url.URL, error))(nil)}
I0429 11:58:35.335671   54098 cert_rotation.go:137] Starting client certificate rotation controller
I0429 11:58:35.340426   54098 round_trippers.go:422] GET https://10.211.55.10:8443/apis/apps/v1/namespaces/kube-system/deployments/coredns/scale
I0429 11:58:35.340453   54098 round_trippers.go:429] Request Headers:
I0429 11:58:35.340461   54098 round_trippers.go:433]     Accept: application/json, */*
I0429 11:58:35.340467   54098 round_trippers.go:433]     User-Agent: minikube/v0.0.0 (darwin/amd64) kubernetes/$Format
I0429 11:58:35.354471   54098 round_trippers.go:448] Response Status: 200 OK in 13 milliseconds
I0429 11:58:35.355074   54098 round_trippers.go:422] PUT https://10.211.55.10:8443/apis/apps/v1/namespaces/kube-system/deployments/coredns/scale
I0429 11:58:35.355095   54098 round_trippers.go:429] Request Headers:
I0429 11:58:35.355103   54098 round_trippers.go:433]     User-Agent: minikube/v0.0.0 (darwin/amd64) kubernetes/$Format
I0429 11:58:35.355111   54098 round_trippers.go:433]     Content-Type: application/json
I0429 11:58:35.355119   54098 round_trippers.go:433]     Accept: application/json, */*
I0429 11:58:35.362032   54098 round_trippers.go:448] Response Status: 200 OK in 6 milliseconds
I0429 11:58:35.866207   54098 round_trippers.go:422] GET https://10.211.55.10:8443/apis/apps/v1/namespaces/kube-system/deployments/coredns/scale
I0429 11:58:35.866257   54098 round_trippers.go:429] Request Headers:
I0429 11:58:35.866273   54098 round_trippers.go:433]     Accept: application/json, */*
I0429 11:58:35.866285   54098 round_trippers.go:433]     User-Agent: minikube/v0.0.0 (darwin/amd64) kubernetes/$Format
I0429 11:58:35.871421   54098 round_trippers.go:448] Response Status: 200 OK in 5 milliseconds
I0429 11:58:35.871555   54098 kapi.go:244] deployment "coredns" in namespace "kube-system" and context "minikube" rescaled to 1
I0429 11:58:35.871610   54098 start.go:200] Will wait 6m0s for node &{Name: IP:10.211.55.10 Port:8443 KubernetesVersion:v1.20.2 ControlPlane:true Worker:true}
I0429 11:58:35.871639   54098 addons.go:328] enableAddons start: toEnable=map[], additional=[]
I0429 11:58:35.871755   54098 addons.go:55] Setting storage-provisioner=true in profile "minikube"
I0429 11:58:35.878913   54098 addons.go:131] Setting addon storage-provisioner=true in "minikube"
I0429 11:58:35.871774   54098 addons.go:55] Setting default-storageclass=true in profile "minikube"
I0429 11:58:35.878829   54098 out.go:157] * Verifying Kubernetes components...
* Verifying Kubernetes components...
W0429 11:58:35.878998   54098 addons.go:140] addon storage-provisioner should already be in state true
I0429 11:58:35.879612   54098 ssh_runner.go:149] Run: sudo systemctl is-active --quiet service kubelet
I0429 11:58:35.879031   54098 addons_storage_classes.go:33] enableOrDisableStorageClasses default-storageclass=true on "minikube"
I0429 11:58:35.879762   54098 host.go:66] Checking if "minikube" exists ...
I0429 11:58:35.880852   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl list minikube --output status --no-header
I0429 11:58:35.881191   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl list minikube --output status --no-header
I0429 11:58:35.896013   54098 loader.go:379] Config loaded from file:  /Users/pierrebeaucamp/.kube/config
I0429 11:58:35.903177   54098 kapi.go:59] client config for minikube: &rest.Config{Host:"https://10.211.55.10:8443", APIPath:"", ContentConfig:rest.ContentConfig{AcceptContentTypes:"", ContentType:"", GroupVersion:(*schema.GroupVersion)(nil), NegotiatedSerializer:runtime.NegotiatedSerializer(nil)}, Username:"", Password:"", BearerToken:"", BearerTokenFile:"", Impersonate:rest.ImpersonationConfig{UserName:"", Groups:[]string(nil), Extra:map[string][]string(nil)}, AuthProvider:, AuthConfigPersister:rest.AuthProviderConfigPersister(nil), ExecProvider:, TLSClientConfig:rest.sanitizedTLSClientConfig{Insecure:false, ServerName:"", CertFile:"/Users/pierrebeaucamp/.minikube/profiles/minikube/client.crt", KeyFile:"/Users/pierrebeaucamp/.minikube/profiles/minikube/client.key", CAFile:"/Users/pierrebeaucamp/.minikube/ca.crt", CertData:[]uint8(nil), KeyData:[]uint8(nil), CAData:[]uint8(nil), NextProtos:[]string(nil)}, UserAgent:"", DisableCompression:false, Transport:http.RoundTripper(nil), WrapTransport:(transport.WrapperFunc)(0x567fdc0), QPS:0, Burst:0, RateLimiter:flowcontrol.RateLimiter(nil), WarningHandler:rest.WarningHandler(nil), Timeout:0, Dial:(func(context.Context, string, string) (net.Conn, error))(nil), Proxy:(func(*http.Request) (*url.URL, error))(nil)}
I0429 11:58:35.908349   54098 api_server.go:48] waiting for apiserver process to appear ...
I0429 11:58:35.908424   54098 ssh_runner.go:149] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0429 11:58:35.921854   54098 api_server.go:68] duration metric: took 50.210461ms to wait for apiserver process to appear ...
I0429 11:58:35.921890   54098 api_server.go:84] waiting for apiserver healthz status ...
I0429 11:58:35.921917   54098 api_server.go:221] Checking apiserver healthz at https://10.211.55.10:8443/healthz ...
I0429 11:58:35.944052   54098 api_server.go:241] https://10.211.55.10:8443/healthz returned 200:
ok
I0429 11:58:35.944211   54098 round_trippers.go:422] GET https://10.211.55.10:8443/version?timeout=32s
I0429 11:58:35.944231   54098 round_trippers.go:429] Request Headers:
I0429 11:58:35.944250   54098 round_trippers.go:433]     Accept: application/json, */*
I0429 11:58:35.944262   54098 round_trippers.go:433]     User-Agent: minikube/v0.0.0 (darwin/amd64) kubernetes/$Format
I0429 11:58:35.945953   54098 round_trippers.go:448] Response Status: 200 OK in 1 milliseconds
I0429 11:58:35.946116   54098 api_server.go:137] control plane version: v1.20.2
I0429 11:58:35.946261   54098 api_server.go:127] duration metric: took 24.358715ms to wait for apiserver health ...
I0429 11:58:35.946302   54098 system_pods.go:42] waiting for kube-system pods to appear ...
I0429 11:58:35.946383   54098 round_trippers.go:422] GET https://10.211.55.10:8443/api/v1/namespaces/kube-system/pods
I0429 11:58:35.946398   54098 round_trippers.go:429] Request Headers:
I0429 11:58:35.946409   54098 round_trippers.go:433]     Accept: application/json, */*
I0429 11:58:35.946423   54098 round_trippers.go:433]     User-Agent: minikube/v0.0.0 (darwin/amd64) kubernetes/$Format
I0429 11:58:35.950671   54098 round_trippers.go:448] Response Status: 200 OK in 4 milliseconds
I0429 11:58:35.958566   54098 system_pods.go:58] 0 kube-system pods found
I0429 11:58:35.958620   54098 retry.go:31] will retry after 263.082536ms: only 0 pod(s) have shown up
I0429 11:58:36.147943   54098 out.go:157]   - Using image gcr.io/k8s-minikube/storage-provisioner:v5
I0429 11:58:36.145806   54098 loader.go:379] Config loaded from file:  /Users/pierrebeaucamp/.kube/config
  - Using image gcr.io/k8s-minikube/storage-provisioner:v5
I0429 11:58:36.148131   54098 addons.go:261] installing /etc/kubernetes/addons/storage-provisioner.yaml
I0429 11:58:36.148249   54098 ssh_runner.go:316] scp memory --> /etc/kubernetes/addons/storage-provisioner.yaml (2676 bytes)
I0429 11:58:36.148301   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl list minikube --output status --no-header
I0429 11:58:36.157452   54098 kapi.go:59] client config for minikube: &rest.Config{Host:"https://10.211.55.10:8443", APIPath:"", ContentConfig:rest.ContentConfig{AcceptContentTypes:"", ContentType:"", GroupVersion:(*schema.GroupVersion)(nil), NegotiatedSerializer:runtime.NegotiatedSerializer(nil)}, Username:"", Password:"", BearerToken:"", BearerTokenFile:"", Impersonate:rest.ImpersonationConfig{UserName:"", Groups:[]string(nil), Extra:map[string][]string(nil)}, AuthProvider:, AuthConfigPersister:rest.AuthProviderConfigPersister(nil), ExecProvider:, TLSClientConfig:rest.sanitizedTLSClientConfig{Insecure:false, ServerName:"", CertFile:"/Users/pierrebeaucamp/.minikube/profiles/minikube/client.crt", KeyFile:"/Users/pierrebeaucamp/.minikube/profiles/minikube/client.key", CAFile:"/Users/pierrebeaucamp/.minikube/ca.crt", CertData:[]uint8(nil), KeyData:[]uint8(nil), CAData:[]uint8(nil), NextProtos:[]string(nil)}, UserAgent:"", DisableCompression:false, Transport:http.RoundTripper(nil), WrapTransport:(transport.WrapperFunc)(0x567fdc0), QPS:0, Burst:0, RateLimiter:flowcontrol.RateLimiter(nil), WarningHandler:rest.WarningHandler(nil), Timeout:0, Dial:(func(context.Context, string, string) (net.Conn, error))(nil), Proxy:(func(*http.Request) (*url.URL, error))(nil)}
I0429 11:58:36.160954   54098 round_trippers.go:422] GET https://10.211.55.10:8443/apis/storage.k8s.io/v1/storageclasses
I0429 11:58:36.160985   54098 round_trippers.go:429] Request Headers:
I0429 11:58:36.160999   54098 round_trippers.go:433]     Accept: application/json, */*
I0429 11:58:36.161010   54098 round_trippers.go:433]     User-Agent: minikube/v0.0.0 (darwin/amd64) kubernetes/$Format
I0429 11:58:36.164534   54098 round_trippers.go:448] Response Status: 200 OK in 3 milliseconds
I0429 11:58:36.165229   54098 addons.go:131] Setting addon default-storageclass=true in "minikube"
W0429 11:58:36.165250   54098 addons.go:140] addon default-storageclass should already be in state true
I0429 11:58:36.165264   54098 host.go:66] Checking if "minikube" exists ...
I0429 11:58:36.165747   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl list minikube --output status --no-header
I0429 11:58:36.224881   54098 round_trippers.go:422] GET https://10.211.55.10:8443/api/v1/namespaces/kube-system/pods
I0429 11:58:36.224939   54098 round_trippers.go:429] Request Headers:
I0429 11:58:36.224956   54098 round_trippers.go:433]     Accept: application/json, */*
I0429 11:58:36.224971   54098 round_trippers.go:433]     User-Agent: minikube/v0.0.0 (darwin/amd64) kubernetes/$Format
I0429 11:58:36.229700   54098 round_trippers.go:448] Response Status: 200 OK in 4 milliseconds
I0429 11:58:36.231673   54098 system_pods.go:58] 4 kube-system pods found
I0429 11:58:36.231723   54098 system_pods.go:60] "etcd-minikube" [ac315b64-93d5-4bcc-b4d5-2810d8763769] Pending
I0429 11:58:36.231741   54098 system_pods.go:60] "kube-apiserver-minikube" [65b71f3e-39eb-4136-8a0e-d5ad9f77f7a8] Pending
I0429 11:58:36.231762   54098 system_pods.go:60] "kube-controller-manager-minikube" [09715fed-aed4-4492-8c11-52eb395e00dc] Pending
I0429 11:58:36.231776   54098 system_pods.go:60] "kube-scheduler-minikube" [ed1d8fc4-b61c-4275-901a-ecf5013b4b6c] Pending
I0429 11:58:36.231791   54098 system_pods.go:73] duration metric: took 285.485116ms to wait for pod list to return data ...
I0429 11:58:36.231809   54098 kubeadm.go:543] duration metric: took 360.185943ms to wait for : map[apiserver:true system_pods:true] ...
I0429 11:58:36.231841   54098 node_conditions.go:102] verifying NodePressure condition ...
I0429 11:58:36.231912   54098 round_trippers.go:422] GET https://10.211.55.10:8443/api/v1/nodes
I0429 11:58:36.231936   54098 round_trippers.go:429] Request Headers:
I0429 11:58:36.231951   54098 round_trippers.go:433]     Accept: application/json, */*
I0429 11:58:36.231964   54098 round_trippers.go:433]     User-Agent: minikube/v0.0.0 (darwin/amd64) kubernetes/$Format
I0429 11:58:36.235354   54098 round_trippers.go:448] Response Status: 200 OK in 3 milliseconds
I0429 11:58:36.237709   54098 node_conditions.go:122] node storage ephemeral capacity is 17784752Ki
I0429 11:58:36.237765   54098 node_conditions.go:123] node cpu capacity is 2
I0429 11:58:36.237801   54098 node_conditions.go:105] duration metric: took 5.938713ms to run NodePressure ...
I0429 11:58:36.237825   54098 start.go:205] waiting for startup goroutines ...
I0429 11:58:36.353676   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl list -i minikube
I0429 11:58:36.370338   54098 addons.go:261] installing /etc/kubernetes/addons/storageclass.yaml
I0429 11:58:36.370375   54098 ssh_runner.go:316] scp memory --> /etc/kubernetes/addons/storageclass.yaml (271 bytes)
I0429 11:58:36.370450   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl list minikube --output status --no-header
I0429 11:58:36.571062   54098 main.go:126] libmachine: Found lease: 10.211.55.10 for MAC: 001C42E8F8CB, expiring at 1619713674, leased for 1800 s.

I0429 11:58:36.571109   54098 main.go:126] libmachine: Found IP lease: 10.211.55.10 for MAC address 001C42E8F8CB

I0429 11:58:36.571127   54098 sshutil.go:53] new ssh client: &{IP:10.211.55.10 Port:22 SSHKeyPath:/Users/pierrebeaucamp/.minikube/machines/minikube/id_rsa Username:docker}
I0429 11:58:36.574870   54098 main.go:126] libmachine: executing: /usr/local/bin/prlctl list -i minikube
I0429 11:58:36.627616   54098 ssh_runner.go:149] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.2/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml
I0429 11:58:36.804646   54098 main.go:126] libmachine: Found lease: 10.211.55.10 for MAC: 001C42E8F8CB, expiring at 1619713674, leased for 1800 s.

I0429 11:58:36.804688   54098 main.go:126] libmachine: Found IP lease: 10.211.55.10 for MAC address 001C42E8F8CB

I0429 11:58:36.804707   54098 sshutil.go:53] new ssh client: &{IP:10.211.55.10 Port:22 SSHKeyPath:/Users/pierrebeaucamp/.minikube/machines/minikube/id_rsa Username:docker}
I0429 11:58:36.853305   54098 ssh_runner.go:149] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.2/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml
I0429 11:58:37.091157   54098 out.go:157] * Enabled addons: storage-provisioner, default-storageclass
* Enabled addons: storage-provisioner, default-storageclass
I0429 11:58:37.091612   54098 addons.go:330] enableAddons completed in 1.220054014s
I0429 11:58:37.718107   54098 start.go:460] kubectl: 1.21.0, cluster: 1.20.2 (minor skew: 1)
I0429 11:58:37.721982   54098 out.go:157] * Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default
* Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default
afbjorklund commented 3 years ago

minikube tunnel will normally give you a random port, as seen in minikube service.

$ minikube ip
192.168.39.199
$ minikube service hello-minikube1
|-----------|-----------------|-------------|-----------------------------|
| NAMESPACE |      NAME       | TARGET PORT |             URL             |
|-----------|-----------------|-------------|-----------------------------|
| default   | hello-minikube1 |        8080 | http://192.168.39.199:31737 |
|-----------|-----------------|-------------|-----------------------------|
🎉  Opening service default/hello-minikube1 in default browser...

With this driver (with a real IP), it is quite similar to running the service with NodePort.

There is no good way to fake the broken network yet, it's driver.NeedsPortForward

pierrebeaucamp commented 3 years ago

Sorry, I'm not sure if I follow you.

When running minikube using the hyperkit or virtualbox drivers, it behaves as expected: minikube tunnel seems to create routes from my local machine to the cluster CIDR, so I can access various LoadBalancer services using a cluster internal IP (shows up under the externalIPs field of a service and is distinctly different from the IP returned by minikube ip).

However, this seems to be broken when using Parallels as the driver. I'm assuming this is a bug since I don't see any remarks on the driver page.

Unfortunately, I'm not sure what mean by "There is no good way to fake the broken network yet, it's driver.NeedsPortForward"

afbjorklund commented 3 years ago

Unfortunately, I'm not sure what mean by "There is no good way to fake the broken network yet, it's driver.NeedsPortForward"

Normally minikube tunnel is used by the Docker Desktop driver, which is hard to emulate on other systems such as Linux

// NeedsPortForward returns true if driver is unable provide direct IP connectivity
func NeedsPortForward(name string) bool {
        if !IsKIC(name) {
                return false
        }
        if oci.IsExternalDaemonHost(name) {
                return true
        }
        // Docker for Desktop
        return runtime.GOOS == "darwin" || runtime.GOOS == "windows" || detect.IsMicrosoftWSL()
}

It could be something specific to parallels on macos, so please disregard the comment if it is indeed working for other drivers.

If you run with --alsologtostderr, you should see the individual operations being performed (both for start and for tunnel)

afbjorklund commented 3 years ago

@pierrebeaucamp : what I meant was that there are the two ways of setting up the tunnels, either routes or ssh

It seems that the networking in the Parallels driver is different than in the other ones, maybe it is missing the device ?

For VirtualBox we have one eth0 NAT interface (without an IP) and one eth1 HostOnly interface (with the IP) So the ssh connection is tunneled in through the first one, and the ingress is routed in through the second one.

If there is indeed no virtual network interface, then minikube tunnel could have used the other way (port forwarding)

But I haven't verified it myself.

https://kb.parallels.com/4948

pierrebeaucamp commented 3 years ago

Here is the more detailed output of the tunnel command

mininkube tunnel --alsologstostderr 
I0507 15:01:10.020313   38452 out.go:278] Setting OutFile to fd 1 ...
I0507 15:01:10.020477   38452 out.go:330] isatty.IsTerminal(1) = false
I0507 15:01:10.020483   38452 out.go:291] Setting ErrFile to fd 2...
I0507 15:01:10.020489   38452 out.go:330] isatty.IsTerminal(2) = false
I0507 15:01:10.020595   38452 root.go:317] Updating PATH: /Users/pierrebeaucamp/.minikube/bin
I0507 15:01:10.021039   38452 mustload.go:65] Loading cluster: minikube
I0507 15:01:10.022107   38452 main.go:126] libmachine: executing: /usr/local/bin/prlctl list minikube --output status --no-header
I0507 15:01:10.197020   38452 host.go:66] Checking if "minikube" exists ...
I0507 15:01:10.197346   38452 api_server.go:146] Checking apiserver status ...
I0507 15:01:10.197424   38452 ssh_runner.go:149] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0507 15:01:10.197486   38452 main.go:126] libmachine: executing: /usr/local/bin/prlctl list minikube --output status --no-header
I0507 15:01:10.365427   38452 main.go:126] libmachine: executing: /usr/local/bin/prlctl list -i minikube
I0507 15:01:10.546209   38452 main.go:126] libmachine: Found lease: 10.211.55.12 for MAC: 001C42F7473B, expiring at 1620415646, leased for 1800 s.

I0507 15:01:10.546235   38452 main.go:126] libmachine: Found IP lease: 10.211.55.12 for MAC address 001C42F7473B

I0507 15:01:10.546250   38452 sshutil.go:53] new ssh client: &{IP:10.211.55.12 Port:22 SSHKeyPath:/Users/pierrebeaucamp/.minikube/machines/minikube/id_rsa Username:docker}
I0507 15:01:10.603017   38452 ssh_runner.go:149] Run: sudo egrep ^[0-9]+:freezer: /proc/3898/cgroup
I0507 15:01:10.608955   38452 api_server.go:162] apiserver freezer: "10:freezer:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8931d01e4ffe12d0fd564ad495b05402.slice/docker-f0ea4b4a23c4e8e637c6cc2527fe4e0bcbdd3cfab550b4d9d2415d6ce1a9df9a.scope"
I0507 15:01:10.609101   38452 ssh_runner.go:149] Run: sudo cat /sys/fs/cgroup/freezer/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8931d01e4ffe12d0fd564ad495b05402.slice/docker-f0ea4b4a23c4e8e637c6cc2527fe4e0bcbdd3cfab550b4d9d2415d6ce1a9df9a.scope/freezer.state
I0507 15:01:10.618556   38452 api_server.go:184] freezer state: "THAWED"
I0507 15:01:10.618760   38452 api_server.go:221] Checking apiserver healthz at https://10.211.55.12:8443/healthz ...
I0507 15:01:10.634930   38452 api_server.go:241] https://10.211.55.12:8443/healthz returned 200:
ok
I0507 15:01:10.634962   38452 tunnel.go:57] Checking for tunnels to cleanup...
I0507 15:01:10.645676   38452 host.go:66] Checking if "minikube" exists ...
I0507 15:01:10.645952   38452 main.go:126] libmachine: executing: /usr/local/bin/prlctl list minikube --output status --no-header
I0507 15:01:10.819259   38452 main.go:126] libmachine: executing: /usr/local/bin/prlctl list minikube --output status --no-header
I0507 15:01:10.989681   38452 main.go:126] libmachine: executing: /usr/local/bin/prlctl list -i minikube
I0507 15:01:11.172307   38452 main.go:126] libmachine: Found lease: 10.211.55.12 for MAC: 001C42F7473B, expiring at 1620415646, leased for 1800 s.

I0507 15:01:11.172336   38452 main.go:126] libmachine: Found IP lease: 10.211.55.12 for MAC address 001C42F7473B

I0507 15:01:11.172421   38452 tunnel_manager.go:71] Setting up tunnel...
I0507 15:01:11.172489   38452 tunnel_manager.go:81] Started minikube tunnel.
I0507 15:01:16.176956   38452 host.go:66] Checking if "minikube" exists ...
I0507 15:01:16.177745   38452 main.go:126] libmachine: executing: /usr/local/bin/prlctl list minikube --output status --no-header
I0507 15:01:16.362174   38452 route_darwin.go:187] preparing DNS forwarding config in "/etc/resolver/cluster.local":
nameserver 10.96.0.10
search_order 1
I0507 15:01:16.422098   38452 route_darwin.go:226] DNS forwarding now configured in "/etc/resolver/cluster.local"
I0507 15:01:16.422315   38452 route_darwin.go:49] Adding route for CIDR 10.96.0.0/12 to gateway 10.211.55.12
I0507 15:01:16.422414   38452 route_darwin.go:51] About to run command: [sudo route -n add 10.96.0.0/12 10.211.55.12]
I0507 15:01:16.460580   38452 route_darwin.go:58] add net 10.96.0.0: gateway 10.211.55.12
I0507 15:01:16.487184   38452 loadbalancer_patcher.go:80] hello-minikube1 is type LoadBalancer.
I0507 15:01:16.497987   38452 loadbalancer_patcher.go:122] Patched hello-minikube1 with IP 10.98.10.229
Status: 
    machine: minikube
    pid: 38452
    route: 10.96.0.0/12 -> 10.211.55.12
    minikube: Running
    services: [hello-minikube1]
    errors: 
        minikube: no errors
        router: no errors
        loadbalancer emulator: no errors

The route seems to be added correctly to the route table (selected output from netstat -rn):

10.96/12           10.211.55.12       UGSc       bridge1                                                                                                                                                     
10.211.55/24       link#12            UC         bridge1      !                                                                                                                                              
10.211.55.12       0.1c.42.f7.47.3b   UHLWIi     bridge1   1159
k8s-triage-robot commented 2 years ago

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

pierrebeaucamp commented 2 years ago

/remove-lifecycle stale

sharifelgamal commented 2 years ago

Unfortunately, none of our maintainers have an easy way to test minikube with the parallels driver, and therefore it's being maintained on a best effort basis. If anyone has a way to replicate this and fix it, we would love the help.

k8s-triage-robot commented 2 years ago

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot commented 2 years ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

caerulescens commented 5 months ago

/assign

caerulescens commented 5 months ago

I can investigate this; I will update what I find.

caerulescens commented 5 months ago

Parallels is fully supported as a virtualization driver for Minikube on Intel-based Mac computers. However, it is important to note that using Parallels as a virtualization driver with the ARM64 architecture is not currently supported.

Parallels notes that driver does not support Apple Silicon yet; see here. They recommend using the docker driver. I would need to test this on my 2015 MacBook Pro, which has an Intel processor. It's currently loaned out to someone, so I will attempt to start the driver in a week or so.