RA489
commented
2 years ago /kind support
Closed greenchapter closed 1 year ago
RA489
commented
2 years ago /kind support
carl-reverb
commented
2 years ago Attempted exactly this procedure on M1 Mac: MacBook Pro (16-inch, 2021), with slight modification:
podman machine init --cpus 2 --memory 2048 --disk-size 20 --image-path nextpodman machine startpodman system connection default podman-machine-default-rootminikube start --driver=podman --container-runtime=cri-oProximal output (for searchers)
😄 minikube v1.24.0 on Darwin 12.0.1 (arm64)
✨ Using the podman (experimental) driver based on user configuration
👍 Starting control plane node minikube in cluster minikube
🚜 Pulling base image ...
E1124 09:23:52.515427 2826 cache.go:201] Error downloading kic artifacts: not yet implemented, see issue #8426
🔥 Creating podman container (CPUs=2, Memory=1953MB) .../ 2021/11/24 09:24:18 tcpproxy: for incoming conn 127.0.0.1:49540, error dialing "192.168.127.2:39067": connect tcp 192.168.127.2:39067: connection was refused
\ 2021/11/24 09:24:21 tcpproxy: for incoming conn 127.0.0.1:49541, error dialing "192.168.127.2:39067": connect tcp 192.168.127.2:39067: connection was refused
[... much of the same output elided]
- 2021/11/24 09:29:54 tcpproxy: for incoming conn 127.0.0.1:49749, error dialing "192.168.127.2:39067": connect tcp 192.168.127.2:39067: connection was refused
✋ Stopping node "minikube" ...
🛑 Powering off "minikube" via SSH ...
2021/11/24 09:29:55 tcpproxy: for incoming conn 127.0.0.1:49759, error dialing "192.168.127.2:39067": connect tcp 192.168.127.2:39067: connection was refused
ERRO[0415] accept tcp [::]:42671: use of closed network connection
ERRO[0415] accept tcp [::]:39067: use of closed network connection
ERRO[0415] accept tcp [::]:44001: use of closed network connection
ERRO[0415] accept tcp [::]:41557: use of closed network connection
ERRO[0415] accept tcp [::]:41943: use of closed network connection
🔥 Deleting "minikube" in podman ...
🤦 StartHost failed, but will try again: creating host: create host timed out in 360.000000 seconds
🔥 Creating podman container (CPUs=2, Memory=1953MB) ...
😿 Failed to start podman container. Running "minikube delete" may fix it: creating host: create: creating: setting up container node: creating volume for minikube container: podman volume create minikube --label name.minikube.sigs.k8s.io=minikube --label created_by.minikube.sigs.k8s.io=true: exit status 125
stdout:
stderr:
Error: volume with name minikube already exists: volume already exists
❌ Exiting due to GUEST_PROVISION: Failed to start host: creating host: create: creating: setting up container node: creating volume for minikube container: podman volume create minikube --label name.minikube.sigs.k8s.io=minikube --label created_by.minikube.sigs.k8s.io=true: exit status 125
stdout:
stderr:
Error: volume with name minikube already exists: volume already exists
greenchapter
commented
2 years ago Is there someone who has a running minikube installtion on Apple Silicon M1 without the Docker Environment?
Cluas
commented
2 years ago Is there someone who has a running minikube installtion on Apple Silicon M1 without the Docker Environment?
I finally chose rancher-desktop.
yllekz
commented
2 years ago Also curious on the answer to this. I wanted to use this as part of the "drop-in replacement for Docker Desktop" as this guide suggests but it seems this isn't compatible with the M1 at all in its current state?
duttonw
commented
2 years ago Yep, also looking at drop in replacements as 'Docker Desktop' is now a no go for adhoc pdev for docker-compose helper scripts for ui dev's. The guide https://dhwaneetbhatt.com/blog/run-docker-without-docker-desktop-on-macos is great for intel cpu's but fall on its face for m1 arm64.
There seems to be nothing truely available to run containers on M1 easily for the non-techy users..
Cluas
commented
2 years ago Yep, also looking at drop in replacements as 'Docker Desktop' is now a no go for adhoc pdev for docker-compose helper scripts for ui dev's. The guide https://dhwaneetbhatt.com/blog/run-docker-without-docker-desktop-on-macos is great for intel cpu's but fall on its face for m1 arm64.
There seems to be nothing truely available to run containers on M1 easily for the non-techy users..
Have a look at rancher-desktop now supports m1, https://github.com/rancher-sandbox/rancher-desktop/issues/956, which is my current alternative
greenchapter
commented
2 years ago Yep, also looking at drop in replacements as 'Docker Desktop' is now a no go for adhoc pdev for docker-compose helper scripts for ui dev's. The guide https://dhwaneetbhatt.com/blog/run-docker-without-docker-desktop-on-macos is great for intel cpu's but fall on its face for m1 arm64. There seems to be nothing truely available to run containers on M1 easily for the non-techy users..
Have a look at rancher-desktop now supports m1, rancher-sandbox/rancher-desktop#956, which is my current alternative
Thank you very much that looks really good and I don't had Rancher Desktop in mind.
The thing is I actually want to prevent a desktop application like Docker, I want to use a CLI tooling like minikube. regarding the notes it looks like that the verion 0.7.0 currently isn't running completely on aarch64.
Highlighted Features
- Apple Silicon (M1) Support: Rancher Desktop can now be installed on Apple Silicon (M1). In the downloads, choose the aarch64 version to get Apple Silicon support. Note: this version does require Rosetta 2 as some of the components needed by Rancher Desktop don’t yet have native builds.
Hopefully we get a M1/AARCH64 solution for minikube 🙏 @RA489
rfay
commented
2 years ago colima (HEAD) with lima/docker works perfectly on arm64. Haven't tried it with kube. I also have mostly good experiences with Rancher Desktop but the bind-mounting doesn't really work.
greenchapter
commented
2 years ago colima (HEAD) with lima/docker works perfectly on arm64. Haven't tried it with kube. I also have mostly good experiences with Rancher Desktop but the bind-mounting doesn't really work.
Oh @rfay that looks really good 🤩
I did a short brew install colima and afterwards a started a kubernetes cluster with colima start --with-kubernetes. In my first testing it runs a k3s cluster on top of qemu, which looks on the first sight very good actually an my M1 machine.
hcguersoy
commented
2 years ago @greenchapter Regarding colima and QEMU : does it start an intel based VM? Or one based on arm64 ?
cdbattags
commented
2 years ago Attempted exactly this procedure on M1 Mac:
MacBook Pro (16-inch, 2021), with slight modification:
podman machine init --cpus 2 --memory 2048 --disk-size 20 --image-path nextpodman machine startpodman system connection default podman-machine-default-rootminikube start --driver=podman --container-runtime=cri-oProximal output (for searchers)
😄 minikube v1.24.0 on Darwin 12.0.1 (arm64) ✨ Using the podman (experimental) driver based on user configuration 👍 Starting control plane node minikube in cluster minikube 🚜 Pulling base image ... E1124 09:23:52.515427 2826 cache.go:201] Error downloading kic artifacts: not yet implemented, see issue #8426 🔥 Creating podman container (CPUs=2, Memory=1953MB) .../ 2021/11/24 09:24:18 tcpproxy: for incoming conn 127.0.0.1:49540, error dialing "192.168.127.2:39067": connect tcp 192.168.127.2:39067: connection was refused \ 2021/11/24 09:24:21 tcpproxy: for incoming conn 127.0.0.1:49541, error dialing "192.168.127.2:39067": connect tcp 192.168.127.2:39067: connection was refused [... much of the same output elided] - 2021/11/24 09:29:54 tcpproxy: for incoming conn 127.0.0.1:49749, error dialing "192.168.127.2:39067": connect tcp 192.168.127.2:39067: connection was refused ✋ Stopping node "minikube" ... 🛑 Powering off "minikube" via SSH ... 2021/11/24 09:29:55 tcpproxy: for incoming conn 127.0.0.1:49759, error dialing "192.168.127.2:39067": connect tcp 192.168.127.2:39067: connection was refused ERRO[0415] accept tcp [::]:42671: use of closed network connection ERRO[0415] accept tcp [::]:39067: use of closed network connection ERRO[0415] accept tcp [::]:44001: use of closed network connection ERRO[0415] accept tcp [::]:41557: use of closed network connection ERRO[0415] accept tcp [::]:41943: use of closed network connection 🔥 Deleting "minikube" in podman ... 🤦 StartHost failed, but will try again: creating host: create host timed out in 360.000000 seconds 🔥 Creating podman container (CPUs=2, Memory=1953MB) ... 😿 Failed to start podman container. Running "minikube delete" may fix it: creating host: create: creating: setting up container node: creating volume for minikube container: podman volume create minikube --label name.minikube.sigs.k8s.io=minikube --label created_by.minikube.sigs.k8s.io=true: exit status 125 stdout: stderr: Error: volume with name minikube already exists: volume already exists ❌ Exiting due to GUEST_PROVISION: Failed to start host: creating host: create: creating: setting up container node: creating volume for minikube container: podman volume create minikube --label name.minikube.sigs.k8s.io=minikube --label created_by.minikube.sigs.k8s.io=true: exit status 125 stdout: stderr: Error: volume with name minikube already exists: volume already exists
I'm having the exact same issue as described by @carl-reverb. Any help would be greatly appreciated. I talked to @baude and he says the driver isn't managed by the Podman folks?
Looks to be an ssh configuration issue I believe?
My specific log output:
I0204 16:28:40.772222 12741 out.go:297] Setting OutFile to fd 1 ...
I0204 16:28:40.772377 12741 out.go:349] isatty.IsTerminal(1) = true
I0204 16:28:40.772382 12741 out.go:310] Setting ErrFile to fd 2...
I0204 16:28:40.772386 12741 out.go:349] isatty.IsTerminal(2) = true
I0204 16:28:40.772451 12741 root.go:315] Updating PATH: /Users/cdbattags/.minikube/bin
I0204 16:28:40.773185 12741 out.go:304] Setting JSON to false
I0204 16:28:40.816599 12741 start.go:112] hostinfo: {"hostname":"christians-mbp.lan","uptime":1836103,"bootTime":1642174017,"procs":817,"os":"darwin","platform":"darwin","platformFamily":"Standalone Workstation","platformVersion":"12.0.1","kernelVersion":"21.1.0","kernelArch":"arm64","virtualizationSystem":"","virtualizationRole":"","hostId":"9f598b00-79ef-5d59-8583-527b2cc6cf00"}
W0204 16:28:40.816759 12741 start.go:120] gopshost.Virtualization returned error: not implemented yet
I0204 16:28:40.837315 12741 out.go:176] 😄 minikube v1.25.1 on Darwin 12.0.1 (arm64)
😄 minikube v1.25.1 on Darwin 12.0.1 (arm64)
I0204 16:28:40.837438 12741 notify.go:174] Checking for updates...
I0204 16:28:40.891481 12741 out.go:176] ▪ KUBECONFIG=/Users/cdbattags/.kube/config:
▪ KUBECONFIG=/Users/cdbattags/.kube/config:
I0204 16:28:40.891662 12741 driver.go:344] Setting default libvirt URI to qemu:///system
I0204 16:28:41.012691 12741 podman.go:121] podman version: 3.4.4
I0204 16:28:41.033257 12741 out.go:176] ✨ Using the podman (experimental) driver based on user configuration
✨ Using the podman (experimental) driver based on user configuration
I0204 16:28:41.033283 12741 start.go:280] selected driver: podman
I0204 16:28:41.033291 12741 start.go:795] validating driver "podman" against <nil>
I0204 16:28:41.033304 12741 start.go:806] status for podman: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc:}
I0204 16:28:41.033316 12741 start.go:1498] auto setting extra-config to "kubelet.housekeeping-interval=5m".
I0204 16:28:41.033465 12741 cli_runner.go:133] Run: podman system info --format json
I0204 16:28:41.135713 12741 info.go:285] podman info: {Host:{BuildahVersion:1.23.1 CgroupVersion:v2 Conmon:{Package:conmon-2.0.30-2.fc35.aarch64 Path:/usr/bin/conmon Version:conmon version 2.0.30, commit: } Distribution:{Distribution:fedora Version:35} MemFree:8511893504 MemTotal:10396856320 OCIRuntime:{Name:crun Package:crun-1.4.1-1.fc35.aarch64 Path:/usr/bin/crun Version:crun version 1.4.1
commit: 802613580a3f25a88105ce4b78126202fef51dfb
spec: 1.0.0
+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL} SwapFree:0 SwapTotal:0 Arch:arm64 Cpus:4 Eventlogger:journald Hostname:minikube Kernel:5.15.17-200.fc35.aarch64 Os:linux Rootless:false Uptime:58m 40.35s} Registries:{Search:[docker.io]} Store:{ConfigFile:/etc/containers/storage.conf ContainerStore:{Number:0} GraphDriverName:overlay GraphOptions:{} GraphRoot:/var/lib/containers/storage GraphStatus:{BackingFilesystem:xfs NativeOverlayDiff:false SupportsDType:true UsingMetacopy:true} ImageStore:{Number:1} RunRoot:/run/containers/storage VolumePath:/var/lib/containers/storage/volumes}}
I0204 16:28:41.135950 12741 start_flags.go:286] no existing cluster config was found, will generate one from the flags
I0204 16:28:41.136068 12741 start_flags.go:367] Using suggested 9867MB memory alloc based on sys=65536MB, container=9915MB
I0204 16:28:41.136138 12741 start_flags.go:796] Wait components to verify : map[apiserver:true system_pods:true]
I0204 16:28:41.136158 12741 cni.go:93] Creating CNI manager for ""
I0204 16:28:41.136162 12741 cni.go:160] "podman" driver + crio runtime found, recommending kindnet
I0204 16:28:41.136167 12741 start_flags.go:295] Found "CNI" CNI - setting NetworkPlugin=cni
I0204 16:28:41.136172 12741 start_flags.go:300] config:
{Name:minikube KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.29@sha256:be897edc9ed473a9678010f390a0092f488f6a1c30865f571c3b6388f9f56f9b Memory:9867 CPUs:2 DiskSize:20000 VMDriver: Driver:podman HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.23.1 ClusterName:minikube Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: ExtraOptions:[{Component:kubelet Key:housekeeping-interval Value:5m}] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror:}
I0204 16:28:41.177463 12741 out.go:176] 👍 Starting control plane node minikube in cluster minikube
👍 Starting control plane node minikube in cluster minikube
I0204 16:28:41.177510 12741 cache.go:120] Beginning downloading kic base image for podman with crio
I0204 16:28:41.196510 12741 out.go:176] 🚜 Pulling base image ...
🚜 Pulling base image ...
I0204 16:28:41.196555 12741 preload.go:132] Checking if preload exists for k8s version v1.23.1 and runtime crio
I0204 16:28:41.196559 12741 cache.go:148] Downloading gcr.io/k8s-minikube/kicbase:v0.0.29@sha256:be897edc9ed473a9678010f390a0092f488f6a1c30865f571c3b6388f9f56f9b to local cache
I0204 16:28:41.196618 12741 preload.go:148] Found local preload: /Users/cdbattags/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v16-v1.23.1-cri-o-overlay-arm64.tar.lz4
I0204 16:28:41.196629 12741 cache.go:57] Caching tarball of preloaded images
I0204 16:28:41.196725 12741 preload.go:174] Found /Users/cdbattags/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v16-v1.23.1-cri-o-overlay-arm64.tar.lz4 in cache, skipping download
I0204 16:28:41.196739 12741 cache.go:60] Finished verifying existence of preloaded tar for v1.23.1 on crio
I0204 16:28:41.196722 12741 image.go:59] Checking for gcr.io/k8s-minikube/kicbase:v0.0.29@sha256:be897edc9ed473a9678010f390a0092f488f6a1c30865f571c3b6388f9f56f9b in local cache directory
I0204 16:28:41.196816 12741 image.go:62] Found gcr.io/k8s-minikube/kicbase:v0.0.29@sha256:be897edc9ed473a9678010f390a0092f488f6a1c30865f571c3b6388f9f56f9b in local cache directory, skipping pull
I0204 16:28:41.196827 12741 image.go:103] gcr.io/k8s-minikube/kicbase:v0.0.29@sha256:be897edc9ed473a9678010f390a0092f488f6a1c30865f571c3b6388f9f56f9b exists in cache, skipping pull
I0204 16:28:41.196836 12741 cache.go:151] successfully saved gcr.io/k8s-minikube/kicbase:v0.0.29@sha256:be897edc9ed473a9678010f390a0092f488f6a1c30865f571c3b6388f9f56f9b as a tarball
I0204 16:28:41.196957 12741 profile.go:147] Saving config to /Users/cdbattags/.minikube/profiles/minikube/config.json ...
I0204 16:28:41.196988 12741 lock.go:35] WriteFile acquiring /Users/cdbattags/.minikube/profiles/minikube/config.json: {Name:mkd321d07bcf4787db3d0ee0fe251b2d1337beb3 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
E0204 16:28:41.215817 12741 cache.go:203] Error downloading kic artifacts: not yet implemented, see issue #8426
I0204 16:28:41.215853 12741 cache.go:208] Successfully downloaded all kic artifacts
I0204 16:28:41.215878 12741 start.go:313] acquiring machines lock for minikube: {Name:mkb6ff83acad3d62d883977c0cfa602b99b93ab3 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0204 16:28:41.215940 12741 start.go:317] acquired machines lock for "minikube" in 47.167µs
I0204 16:28:41.215965 12741 start.go:89] Provisioning new machine with config: &{Name:minikube KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.29@sha256:be897edc9ed473a9678010f390a0092f488f6a1c30865f571c3b6388f9f56f9b Memory:9867 CPUs:2 DiskSize:20000 VMDriver: Driver:podman HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.23.1 ClusterName:minikube Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: ExtraOptions:[{Component:kubelet Key:housekeeping-interval Value:5m}] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.23.1 ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror:} &{Name: IP: Port:8443 KubernetesVersion:v1.23.1 ControlPlane:true Worker:true}
I0204 16:28:41.216061 12741 start.go:126] createHost starting for "" (driver="podman")
I0204 16:28:41.254988 12741 out.go:203] 🔥 Creating podman container (CPUs=2, Memory=9867MB) ...
🔥 Creating podman container (CPUs=2, Memory=9867MB) ...| I0204 16:28:41.255182 12741 start.go:160] libmachine.API.Create for "minikube" (driver="podman")
I0204 16:28:41.255207 12741 client.go:168] LocalClient.Create starting
I0204 16:28:41.255314 12741 main.go:130] libmachine: Reading certificate data from /Users/cdbattags/.minikube/certs/ca.pem
I0204 16:28:41.255355 12741 main.go:130] libmachine: Decoding PEM data...
I0204 16:28:41.255367 12741 main.go:130] libmachine: Parsing certificate...
I0204 16:28:41.255409 12741 main.go:130] libmachine: Reading certificate data from /Users/cdbattags/.minikube/certs/cert.pem
I0204 16:28:41.255436 12741 main.go:130] libmachine: Decoding PEM data...
I0204 16:28:41.255442 12741 main.go:130] libmachine: Parsing certificate...
I0204 16:28:41.255979 12741 cli_runner.go:133] Run: podman network inspect minikube --format "{{range .plugins}}{{if eq .type "bridge"}}{{(index (index .ipam.ranges 0) 0).subnet}},{{(index (index .ipam.ranges 0) 0).gateway}}{{end}}{{end}}"
/ W0204 16:28:41.356346 12741 cli_runner.go:180] podman network inspect minikube --format "{{range .plugins}}{{if eq .type "bridge"}}{{(index (index .ipam.ranges 0) 0).subnet}},{{(index (index .ipam.ranges 0) 0).gateway}}{{end}}{{end}}" returned with exit code 125
I0204 16:28:41.356564 12741 network_create.go:254] running [podman network inspect minikube] to gather additional debugging logs...
I0204 16:28:41.356605 12741 cli_runner.go:133] Run: podman network inspect minikube
- W0204 16:28:41.456851 12741 cli_runner.go:180] podman network inspect minikube returned with exit code 125
I0204 16:28:41.456893 12741 network_create.go:257] error running [podman network inspect minikube]: podman network inspect minikube: exit status 125
stdout:
stderr:
Cannot connect to Podman. Please verify your connection to the Linux system using `podman system connection list`, or try `podman machine init` and `podman machine start` to manage a new Linux VM
Error: unable to connect to Podman. failed to create sshClient: Connection to bastion host (ssh://root@localhost:49525/run/podman/podman.sock) failed.: ssh: handshake failed: ssh: disconnect, reason 2: Too many authentication failures
I0204 16:28:41.456922 12741 network_create.go:259] output of [podman network inspect minikube]:
** stderr **
Cannot connect to Podman. Please verify your connection to the Linux system using `podman system connection list`, or try `podman machine init` and `podman machine start` to manage a new Linux VM
Error: unable to connect to Podman. failed to create sshClient: Connection to bastion host (ssh://root@localhost:49525/run/podman/podman.sock) failed.: ssh: handshake failed: ssh: disconnect, reason 2: Too many authentication failures
** /stderr **
I0204 16:28:41.457095 12741 cli_runner.go:133] Run: podman network inspect podman --format "{{range .plugins}}{{if eq .type "bridge"}}{{(index (index .ipam.ranges 0) 0).subnet}},{{(index (index .ipam.ranges 0) 0).gateway}}{{end}}{{end}}"
I0204 16:28:41.547110 12741 network.go:288] reserving subnet 192.168.49.0 for 1m0s: &{mu:{state:0 sema:0} read:{v:{m:map[] amended:true}} dirty:map[192.168.49.0:0x140007926b0] misses:0}
I0204 16:28:41.547159 12741 network.go:235] using free private subnet 192.168.49.0/24: &{IP:192.168.49.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.49.0/24 Gateway:192.168.49.1 ClientMin:192.168.49.2 ClientMax:192.168.49.254 Broadcast:192.168.49.255 Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:}}
I0204 16:28:41.547171 12741 network_create.go:106] attempt to create podman network minikube 192.168.49.0/24 with gateway 192.168.49.1 and MTU of 0 ...
I0204 16:28:41.547292 12741 cli_runner.go:133] Run: podman network create --driver=bridge --subnet=192.168.49.0/24 --gateway=192.168.49.1 --label=created_by.minikube.sigs.k8s.io=true minikube
\ W0204 16:28:41.654355 12741 cli_runner.go:180] podman network create --driver=bridge --subnet=192.168.49.0/24 --gateway=192.168.49.1 --label=created_by.minikube.sigs.k8s.io=true minikube returned with exit code 125
E0204 16:28:41.654414 12741 network_create.go:95] error while trying to create podman network minikube 192.168.49.0/24: create podman network minikube 192.168.49.0/24 with gateway 192.168.49.1 and MTU of 0: podman network create --driver=bridge --subnet=192.168.49.0/24 --gateway=192.168.49.1 --label=created_by.minikube.sigs.k8s.io=true minikube: exit status 125
stdout:
stderr:
Error: network 192.168.49.0/24 is already being used by a cni configuration
W0204 16:28:41.654564 12741 out.go:241] ❗ Unable to create dedicated network, this might result in cluster IP change after restart: un-retryable: create podman network minikube 192.168.49.0/24 with gateway 192.168.49.1 and MTU of 0: podman network create --driver=bridge --subnet=192.168.49.0/24 --gateway=192.168.49.1 --label=created_by.minikube.sigs.k8s.io=true minikube: exit status 125
stdout:
stderr:
Error: network 192.168.49.0/24 is already being used by a cni configuration
❗ Unable to create dedicated network, this might result in cluster IP change after restart: un-retryable: create podman network minikube 192.168.49.0/24 with gateway 192.168.49.1 and MTU of 0: podman network create --driver=bridge --subnet=192.168.49.0/24 --gateway=192.168.49.1 --label=created_by.minikube.sigs.k8s.io=true minikube: exit status 125
stdout:
stderr:
Error: network 192.168.49.0/24 is already being used by a cni configuration
I0204 16:28:41.654715 12741 cli_runner.go:133] Run: podman ps -a --format {{.Names}}
I0204 16:28:41.749002 12741 cli_runner.go:133] Run: podman volume create minikube --label name.minikube.sigs.k8s.io=minikube --label created_by.minikube.sigs.k8s.io=true
I0204 16:28:41.865998 12741 oci.go:102] Successfully created a podman volume minikube
I0204 16:28:41.866174 12741 cli_runner.go:133] Run: podman run --rm --name minikube-preload-sidecar --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=minikube --entrypoint /usr/bin/test -v minikube:/var gcr.io/k8s-minikube/kicbase:v0.0.29 -d /var/lib
I0204 16:28:42.457001 12741 oci.go:106] Successfully prepared a podman volume minikube
I0204 16:28:42.457088 12741 preload.go:132] Checking if preload exists for k8s version v1.23.1 and runtime crio
I0204 16:28:42.457102 12741 kic.go:179] Starting extracting preloaded images to volume ...
I0204 16:28:42.457244 12741 cli_runner.go:133] Run: podman run --rm --entrypoint /usr/bin/tar -v /Users/cdbattags/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v16-v1.23.1-cri-o-overlay-arm64.tar.lz4:/preloaded.tar:ro -v minikube:/extractDir gcr.io/k8s-minikube/kicbase:v0.0.29 -I lz4 -xf /preloaded.tar -C /extractDir
W0204 16:28:42.548933 12741 cli_runner.go:180] podman run --rm --entrypoint /usr/bin/tar -v /Users/cdbattags/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v16-v1.23.1-cri-o-overlay-arm64.tar.lz4:/preloaded.tar:ro -v minikube:/extractDir gcr.io/k8s-minikube/kicbase:v0.0.29 -I lz4 -xf /preloaded.tar -C /extractDir returned with exit code 125
I0204 16:28:42.548994 12741 kic.go:186] Unable to extract preloaded tarball to volume: podman run --rm --entrypoint /usr/bin/tar -v /Users/cdbattags/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v16-v1.23.1-cri-o-overlay-arm64.tar.lz4:/preloaded.tar:ro -v minikube:/extractDir gcr.io/k8s-minikube/kicbase:v0.0.29 -I lz4 -xf /preloaded.tar -C /extractDir: exit status 125
stdout:
stderr:
Cannot connect to Podman. Please verify your connection to the Linux system using `podman system connection list`, or try `podman machine init` and `podman machine start` to manage a new Linux VM
Error: unable to connect to Podman. failed to create sshClient: Connection to bastion host (ssh://root@localhost:49525/run/podman/podman.sock) failed.: ssh: handshake failed: ssh: disconnect, reason 2: Too many authentication failures
I0204 16:28:42.549129 12741 cli_runner.go:133] Run: podman info --format "'{{json .SecurityOptions}}'"
W0204 16:28:42.654683 12741 cli_runner.go:180] podman info --format "'{{json .SecurityOptions}}'" returned with exit code 125
I0204 16:28:42.654884 12741 cli_runner.go:133] Run: podman run -d -t --privileged --device /dev/fuse --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname minikube --name minikube --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=minikube --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=minikube --volume minikube:/var:exec --memory-swap=9867mb --memory=9867mb --cpus=2 -e container=podman --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 --publish=127.0.0.1::32443 gcr.io/k8s-minikube/kicbase:v0.0.29
I0204 16:28:43.020539 12741 cli_runner.go:133] Run: podman container inspect minikube --format={{.State.Running}}
I0204 16:28:43.181851 12741 cli_runner.go:133] Run: podman container inspect minikube --format={{.State.Status}}
I0204 16:28:43.271128 12741 cli_runner.go:133] Run: podman exec minikube stat /var/lib/dpkg/alternatives/iptables
I0204 16:28:43.510225 12741 oci.go:281] the created container "minikube" has a running status.
I0204 16:28:43.510265 12741 kic.go:210] Creating ssh key for kic: /Users/cdbattags/.minikube/machines/minikube/id_rsa...
I0204 16:28:43.568990 12741 vm_assets.go:163] NewFileAsset: /Users/cdbattags/.minikube/machines/minikube/id_rsa.pub -> /home/docker/.ssh/authorized_keys
I0204 16:28:43.569047 12741 kic_runner.go:191] podman (temp): /Users/cdbattags/.minikube/machines/minikube/id_rsa.pub --> /home/docker/.ssh/authorized_keys (381 bytes)
I0204 16:28:43.573079 12741 kic_runner.go:276] Run: /opt/homebrew/bin/podman exec -i minikube tee /home/docker/.ssh/authorized_keys
I0204 16:28:43.752698 12741 cli_runner.go:133] Run: podman container inspect minikube --format={{.State.Status}}
I0204 16:28:43.890276 12741 kic_runner.go:93] Run: chown docker:docker /home/docker/.ssh/authorized_keys
I0204 16:28:43.890301 12741 kic_runner.go:114] Args: [podman exec --privileged minikube chown docker:docker /home/docker/.ssh/authorized_keys]
I0204 16:28:44.223738 12741 cli_runner.go:133] Run: podman container inspect minikube --format={{.State.Status}}
I0204 16:28:44.381002 12741 machine.go:88] provisioning docker machine ...
I0204 16:28:44.381078 12741 ubuntu.go:169] provisioning hostname "minikube"
I0204 16:28:44.381266 12741 cli_runner.go:133] Run: podman version --format {{.Version}}
I0204 16:28:44.495782 12741 cli_runner.go:133] Run: podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
I0204 16:28:44.680537 12741 main.go:130] libmachine: Using SSH client type: native
I0204 16:28:44.680789 12741 main.go:130] libmachine: &{{{<nil> 0 [] [] []} docker [0x1052cdc10] 0x1052d0a30 <nil> [] 0s} 127.0.0.1 36029 <nil> <nil>}
I0204 16:28:44.680799 12741 main.go:130] libmachine: About to run SSH command:
sudo hostname minikube && echo "minikube" | sudo tee /etc/hostname
2022/02/04 16:28:44 tcpproxy: for incoming conn 127.0.0.1:51001, error dialing "192.168.127.2:36029": connect tcp 192.168.127.2:36029: connection was refused
I0204 16:28:44.681550 12741 main.go:130] libmachine: Error dialing TCP: ssh: handshake failed: read tcp 127.0.0.1:51001->127.0.0.1:36029: read: connection reset by peer
2022/02/04 16:28:47 tcpproxy: for incoming conn 127.0.0.1:51003, error dialing "192.168.127.2:36029": connect tcp 192.168.127.2:36029: connection was refused
I0204 16:28:47.687678 12741 main.go:130] libmachine: Error dialing TCP: ssh: handshake failed: read tcp 127.0.0.1:51003->127.0.0.1:36029: read: connection reset by peer
2022/02/04 16:28:50 tcpproxy: for incoming conn 127.0.0.1:51005, error dialing "192.168.127.2:36029": connect tcp 192.168.127.2:36029: connection was refused
I0204 16:28:50.689023 12741 main.go:130] libmachine: Error dialing TCP: ssh: handshake failed: read tcp 127.0.0.1:51005->127.0.0.1:36029: read: connection reset by peer
2022/02/04 16:28:53 tcpproxy: for incoming conn 127.0.0.1:51007, error dialing "192.168.127.2:36029": connect tcp 192.168.127.2:36029: connection was refused
I0204 16:28:53.692089 12741 main.go:130] libmachine: Error dialing TCP: ssh: handshake failed: read tcp 127.0.0.1:51007->127.0.0.1:36029: read: connection reset by peer
2022/02/04 16:28:56 tcpproxy: for incoming conn 127.0.0.1:51008, error dialing "192.168.127.2:36029": connect tcp 192.168.127.2:36029: connection was refused
I0204 16:28:56.695504 12741 main.go:130] libmachine: Error dialing TCP: ssh: handshake failed: read tcp 127.0.0.1:51008->127.0.0.1:36029: read: connection reset by peer
2022/02/04 16:28:59 tcpproxy: for incoming conn 127.0.0.1:51009, error dialing "192.168.127.2:36029": connect tcp 192.168.127.2:36029: connection was refused
I0204 16:28:59.697838 12741 main.go:130] libmachine: Error dialing TCP: ssh: handshake failed: read tcp 127.0.0.1:51009->127.0.0.1:36029: read: connection reset by peer
spowelljr
commented
2 years ago Just as an update on the HyperKit side, we are making progress towards getting an ARM64 ISO, here's the PR for reference
greenchapter
commented
2 years ago Just as an update on the HyperKit side, we are making progress towards getting an ARM64 ISO, here's the PR for reference
Hopefully it will be merged soon 😍
afbjorklund
commented
2 years ago @spowelljr hyperkit does not support arm64, so the new ISO will have to use some other VM driver (vmware/parallels/qemu2)
afbjorklund
commented
2 years ago Regarding colima and QEMU : does it start an intel based VM? Or one based on arm64 ?
By default, upstream lima starts an intel VM on an intel host and an arm64 VM on an arm64 host:
# Arch: "default", "x86_64", "aarch64".
# 🟢 Builtin default: "default" (corresponds to the host architecture)
arch: nullThe colima distribution might be different, but there is lima Kubernetes support with containerd:
limactl start https://raw.githubusercontent.com/lima-vm/lima/master/examples/k8s.yaml
medyagh
commented
2 years ago Thank everyone for the patience, please track the update in this issue, https://github.com/kubernetes/minikube/issues/9228
k8s-triage-robot
commented
2 years ago The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.
This bot triages issues and PRs according to the following rules:
lifecycle/stale is appliedlifecycle/stale was applied, lifecycle/rotten is appliedlifecycle/rotten was applied, the issue is closedYou can:
/remove-lifecycle stale/lifecycle rotten/closePlease send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale
k8s-triage-robot
commented
2 years ago The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.
This bot triages issues and PRs according to the following rules:
lifecycle/stale is appliedlifecycle/stale was applied, lifecycle/rotten is appliedlifecycle/rotten was applied, the issue is closedYou can:
/remove-lifecycle rotten/closePlease send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle rotten
RA489
commented
2 years ago /remove-lifecycle rotten
k8s-triage-robot
commented
1 year ago The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.
This bot triages issues and PRs according to the following rules:
lifecycle/stale is appliedlifecycle/stale was applied, lifecycle/rotten is appliedlifecycle/rotten was applied, the issue is closedYou can:
/remove-lifecycle stale/lifecycle rotten/closePlease send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale
k8s-triage-robot
commented
1 year ago The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.
This bot triages issues and PRs according to the following rules:
lifecycle/stale is appliedlifecycle/stale was applied, lifecycle/rotten is appliedlifecycle/rotten was applied, the issue is closedYou can:
/remove-lifecycle rotten/closePlease send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle rotten
k8s-triage-robot
commented
1 year ago The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.
This bot triages issues according to the following rules:
lifecycle/stale is appliedlifecycle/stale was applied, lifecycle/rotten is appliedlifecycle/rotten was applied, the issue is closedYou can:
/reopen/remove-lifecycle rottenPlease send feedback to sig-contributor-experience at kubernetes/community.
/close not-planned
k8s-ci-robot
commented
1 year ago @k8s-triage-robot: Closing this issue, marking it as "Not Planned".
Actually I want to run minikube on my Apple Silicon M1 MacBook Pro without using Docker (the Docker integration actually works) actually I focusing on podman and hyperkit but currently nothing of this two drivers will work.
For me it looks like that podman is currently running and I can build images with podman but I got no working hypervisor for minikube. hyperkit looks like is currently not available for macOS Monteray and macOS BigSur.
Steps to reproduce the issue with podman :
podman machine init --cpus 2 --memory 2048 --disk-size 20podman machine startpodman system connection default podman-machine-default-rootminikube start --driver=podman --container-runtime=cri-o --alsologtostderr -v=7Like described here https://minikube.sigs.k8s.io/docs/drivers/podman/
Steps to reproduce the issue with hyperkit :
brew install hyperkitHopefully that is detailed enough and someone can help me to get a running setup without Docker CE / Docker Desktop