kundan2707
commented
1 year ago /kind support
Open hugo-syn opened 1 year ago
kundan2707
commented
1 year ago /kind support
inuyasha82
commented
1 year ago I"m having the same identical error on fedora 38, with podman version 4.5.1. Same output .
Trying to create a minikube profile with the following command:
minikube start --driver=podman --cpus=4 --memory=8192 --kubernetes-version 1.17.1 --disk-size=40g --profile podmanenv --container-runtime=cri-o
afbjorklund
commented
1 year ago It might work better with --container-runtime=containerd (haven't tested)
osiler
commented
9 months ago Also reporting similar issue using rootless podman and the command ~ Podman 4.6.2
minikube start --driver=podman --container-runtime=containerd
smegeath
commented
7 months ago Having the same issue with v1.32.0 of minikube and podman 4.6.1 on Rocky 8.9.
adrian-moisa
commented
7 months ago Got it working. Thanks @osiler! I'm usually keeping deep notes whenever I'm stuck on a critical problem. I'll share here what worked and what not. There's the TLDR and the long read. Hopefully both help you in the future.
TLDR - In short summary, due to unexpected latencies stemming from cache invalidation I mistakenly thought my podman/minikube setup is broken again. I reinstalled everything. this time minikube did not work. I thought it's a versions issues, tried many combos, nothing worked. I tried rolling back podman, qemu, kubectl and got no progress. Eventually I tried without podman driver and it works using qemu driver. It also works if I specify podman driver and containerd runtime. Wasted 12 hours to find this. Over 25 reinstalls. Thanks to @osiler for sharing the --container-runtime=containerd flag
Full description of diagnosis attempts - Hint: this is the long read.
For some unknown reason my minikube stopped working reliably after I had a laptop sleep cycle (went out a bit). The minikube pods were not able to start (I had an odd feeling seeing them struggle so much). I also noticed podman stats not responding, another sign of trouble. I tried rebuilding one of them but I kept noticing what felt to me strange behaviors: Podman builds hanging (I misinterpreted podman behavior). It felt as if the podman builds do not respond. Then I tried to rebuild the base image of flutter in case there's something wrong there. I had the same odd behavior, lots of hanging. I tried running verbose and again I saw little useful info besides the list of docker ignore files.
After many tries I had a feeling there's an issue with the podman setup. So I decided to do a complete reinstall. This turned out later to be a completely undesired choice. After reinstalling podman I tried again building the docker images and I saw the same behavior of hanging a long time. It occurred to me to check the network traffic and I noticed there's some significant amount. So I deduced that there's unreported activity even if not visible in the terminal. Decided to wait it out and eventually it finished and moved on to the building steps.
This means that podman was basically waiting to download the base image. For some reason it does not report in terminal this download stage. After some more review I realized that podman invalidated the original caches of the go and flutter base images because I changed in many places the dockerignore files. Lesson learned. Being used to see fast and cached builds I've mistaken this delay with problems in podman caused potentially by the stand by period. Knowing that in the past similar events were really hard to debug I decided to cut the chase and to a full reinstall hoping to get back to work quickly. This is what led to the actual problem of not being able to start minikube.
As I explained, I thought I'm doing smth good by skipping the long effort of debugging minikube. Turns out that after the new install minikube no longer worked. Everything seemed to work fine up to the point of starting the control pane. This is where it hangs a lot and fails. Renders some diagnosis with hints to what errors could be the root cause. I searched them on google and GPT did not find anything useful. After checking my podman and minikube versions I remember about the issues I had in the past with a broken version of qemu. So I decided to rollback. After some more hassle there I finally managed to revert podman and qemu to what I knew as the last stable versions. Sadly no luck still no change. Minikube still fails. Despite attempting to completely erasing podman and minikube including settings files.
This was what stood out from the logs. Searching on we I couldn't find anything useful. Logs can be reviewed with minikube logs.
[WARNING SystemVerification]: missing optional cgroups: hugetlb
[WARNING SystemVerification]: missing required cgroups: cpuset
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
[WARNING FileContent--proc-sys-net-bridge-bridge-nf-call-iptables]: /proc/sys/net/bridge/bridge-nf-call-iptables does not exist
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higherReinstall Prior Version Of Podman Using podman.rb - No Help
Prior to this event I had an issue with latest qemu breaking podman machine. The solution was to rollback to a prior version of qemu. A few days later the qemu team fixed it. Having this experience I thought this is the same thing again. So I tried to rollback various combinations of podman, qemu, kubectl and minikube. Nothing helped in the end.
Full uninstall procedure Eventually I figured out that all I need to remove is minikube. That's why I sorted by categories the cleanup procedure.
# Minikube
minikube stop --all
minikube delete --all --purge
brew uninstall --force minikube
rm -rf ~/.minikube
# Kubernetes CLI
brew uninstall --force kubectl
rm -rf ~/.kube
# Podman
podman stop -a
podman rm -a
brew uninstall --force podman-desktop
brew uninstall --force podman
rm -rf ~/.local/share/containers
rm -rf ~/.config/containers/
rm ~/.ssh/podman*
# Qemu
brew uninstall --force qemu
rm -rf ~/.qemu
rm -rf ~/.config/qemuRestart procedure Note: You have to prepare your .rb files as instructed in the tutorial if you want to rollback.
brew install podman.rb
brew reinstall qemu.rb
brew install podman-desktop
brew install kubernetes-cli.rb
brew install minikube.rb
# Double check versions
brew list podman
brew list qemu
brew list podman-desktop
brew list kubectl
brew list minikube
# Init podman
podman machine init
podman machine stop
podman machine set --cpus 2 --memory=8192
podman machine start
# Init minikube
minikube start -p myspace --driver=podman --memory=6g
kubectl create namespace myspace
minikube -p myspace addons enable metrics-server
minikube -p myspace addons enable dashboard
minikube -p myspace dashboard &
minikube tunnel -p myspaceSometimes brew links are lost
brew link podman
brew link qemuAttempting Verbose Logs
minikube start -p visualspace --driver=podman --memory=6g --alsologtostderr -v=7 - There's a super verbose flag that can be used when starting minikube. I kept getting this error on repeat many times.
E0220 22:52:00.889626 36203 memcache.go:265] couldn't get current server API group list: Get "https://localhost:8443/api?timeout=32s": dial tcp [::1]:8443: connect: connection refused~/.kube/config - Found this after trying this command kubectl get nodes -v=10 from here: [Kubectl error: memcache.go:265] couldnโt get current server API group list: Get](https://stackoverflow.com/questions/76841889/kubectl-error-memcache-go265-couldn-t-get-current-server-api-group-list-get) I added this folder to be removed by the reset procedure. Same result. No progressTrying Older Kubectl - Did the trick - Tried Again, Could Not Reproduce
After trying an older kubectl it finally worked again. I noticed that kubectl release many weekly updates, surely a recent one broke things. The problem started since I downloaded 1.29.2. I tried a second time the older version. Still no luck
Investigating CNI issues I found a tutorial on the web that suggested to check CNI Troubleshooting the โfailed to create pod sandboxโ error. I looked in the issues-journal.md for the previous problems we had with CNI and found these commands to read the state of the network.
/etc/cni/net.d/1-k8s.conflist does not exist. This file is typically responsible for defining how Kubernetes pods should be networked together. The absence of this file suggests that the necessary CNI network configuration for Kubernetes has not been successfully applied or is missing. I'm not going down the route of trying to patch the system after install. I'd rather have it install correctly from the first try.
minikube ssh
docker@minikube:~$ cat /etc/cni/net.d/1-k8s.conflist
cat /etc/cni/net.d/200-loopback.confcat: /etc/cni/net.d/1-k8s.conflist: No such file or directory { "cniVersion": "1.0.0", "name": "loopback", "type": "loopback" }
- **Enable CNI param - No help** - GPT: Ensure that Minikube is started with the correct network plugin options. Sometimes specifying `--network-plugin=cni` and `--enable-default-cni` during Minikube start-up can help ensure the correct CNI configurations are applied. I tried it, makes no difference.
**Fix1: Qemu Driver instead of podman - WORKS**
I tried running without podman driver. It seems that the entire problem can be fixed by removing the `--driver=podman` param. When doing so, minikube seems to be using qemu as driver. Podman uses qemu as driver as well. Then why bother at all with the podman driver?
- **Do we need local rootless?** - GTP keeps saying that rootless is important. As long as VPS Kubernetes uses smth else to run why does this matter at all? I'm not the main target as a local developer. So rootless local is not such a big win. I guess.
- **No Docker Desktop, No license issue** - Also regarding avoiding docker licenses. That is for docker desktop. We use podman for the local development stuff. Therefore there's no strong reason to use podman driver when running minikube (that I know of).
**Fix2: Containerd instead of CRI-O - WORKS**
- `minikube start --driver=podman --container-runtime=containerd` - Found here: [Fail to start minikube with root-less podman #16919](https://github.com/kubernetes/minikube/issues/16919) - A day ago I I managed to make it work while attempting rollbacks. At one point minikube asked if I want `containerd` or `cri-o`. I know that I chose `containerd` because it is the oldest format on the market since docker. Meaning it's more reliable. The setup worked and I thought I found the right combo of rollbacks. However this was the only time minikube asked which runtime to use. The second day I tried to reinstall all of it again to prove my previous theory of the versions rollback. Again it stopped working. I learned just recently of this param to select container runtime. I don't understand on what basis minikube chooses to ask you or not. It is certain that there is this param by which you can say that you necessarily want containerd. And then it runs ok.
k8s-triage-robot
commented
4 months ago The Kubernetes project currently lacks enough contributors to adequately respond to all issues.
This bot triages un-triaged issues according to the following rules:
lifecycle/stale is appliedlifecycle/stale was applied, lifecycle/rotten is appliedlifecycle/rotten was applied, the issue is closedYou can:
/remove-lifecycle stale/closePlease send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale
waja
commented
4 months ago /remove-lifecycle stale
kuanyui
commented
2 months ago quoted from @adrian-moisa 's reply:
Fix2: Containerd instead of CRI-O - WORKS
minikube start --driver=podman --container-runtime=containerd- Found here: Fail to start minikube with root-less podman #16919 - A day ago I I managed to make it work while attempting rollbacks. At one point minikube asked if I wantcontainerdorcri-o. I know that I chosecontainerdbecause it is the oldest format on the market since docker. Meaning it's more reliable. The setup worked and I thought I found the right combo of rollbacks. However this was the only time minikube asked which runtime to use. The second day I tried to reinstall all of it again to prove my previous theory of the versions rollback. Again it stopped working. I learned just recently of this param to select container runtime. I don't understand on what basis minikube chooses to ask you or not. It is certain that there is this param by which you can say that you necessarily want containerd. And then it runs ok.
Minikube's official document also said that though
For Rootless Podman, it is recommended to set
--container-runtimetocontainerd:minikube start --driver=podman --container-runtime=containerd
But I still get same error and failed to start up:
Business-Wizard
commented
1 month ago Got it working. Thanks @osiler! I'm usually keeping deep notes whenever I'm stuck on a critical problem. I'll share here what worked and what not. There's the TLDR and the long read. Hopefully both help you in the future.
TLDR2, Allowed me to skip reinstalling or convoluted bughunts: Details: arm64 macos, nix-shell env
minikube delete
minikube start --driver podman --container-runtime containerd
What Happened?
I can't start minikube with the podman driver in rootless mode.
Attach the log file
` /stderr I0720 17:52:24.654165 156176 cli_runner.go:164] Run: podman version --format {{.Version}} I0720 17:52:24.689875 156176 cli_runner.go:164] Run: podman network inspect podman --format "{{range .}}{{if eq .Driver "bridge"}}{{(index .Subnets 0).Subnet}},{{(index .Subnets 0).Gateway}}{{end}}{{end}}" I0720 17:52:24.725931 156176 network.go:209] using free private subnet 192.168.49.0/24: &{IP:192.168.49.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.49.0/24 Gateway:192.168.49.1 ClientMin:192.168.49.2 ClientMax:192.168.49.254 Broadcast:192.168.49.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:0xc0005f7960} I0720 17:52:24.725967 156176 network_create.go:123] attempt to create podman network minikube 192.168.49.0/24 with gateway 192.168.49.1 and MTU of 0 ... I0720 17:52:24.726003 156176 cli_runner.go:164] Run: podman network create --driver=bridge --subnet=192.168.49.0/24 --gateway=192.168.49.1 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=minikube minikube I0720 17:52:24.764246 156176 network_create.go:107] podman network minikube 192.168.49.0/24 created I0720 17:52:24.764262 156176 kic.go:117] calculated static IP "192.168.49.2" for the "minikube" container I0720 17:52:24.764316 156176 cli_runner.go:164] Run: podman ps -a --format {{.Names}} I0720 17:52:24.798459 156176 cli_runner.go:164] Run: podman volume create minikube --label name.minikube.sigs.k8s.io=minikube --label created_by.minikube.sigs.k8s.io=true I0720 17:52:24.844224 156176 oci.go:103] Successfully created a podman volume minikube I0720 17:52:24.844264 156176 cli_runner.go:164] Run: podman run --rm --name minikube-preload-sidecar --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=minikube --entrypoint /usr/bin/test -v minikube:/var gcr.io/k8s-minikube/kicbase:v0.0.40 -d /var/lib I0720 17:52:28.170992 156176 cli_runner.go:217] Completed: podman run --rm --name minikube-preload-sidecar --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=minikube --entrypoint /usr/bin/test -v minikube:/var gcr.io/k8s-minikube/kicbase:v0.0.40 -d /var/lib: (3.326690571s) I0720 17:52:28.171027 156176 oci.go:107] Successfully prepared a podman volume minikube I0720 17:52:28.171054 156176 preload.go:132] Checking if preload exists for k8s version v1.27.3 and runtime containerd I0720 17:52:28.171071 156176 kic.go:190] Starting extracting preloaded images to volume ... I0720 17:52:28.171126 156176 cli_runner.go:164] Run: podman run --rm --entrypoint /usr/bin/tar --security-opt label=disable -v /home/user/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.27.3-containerd-overlay2-amd64.tar.lz4:/preloaded.tar:ro -v minikube:/extractDir gcr.io/k8s-minikube/kicbase:v0.0.40 -I lz4 -xf /preloaded.tar -C /extractDir I0720 17:52:39.362900 156176 cli_runner.go:217] Completed: podman run --rm --entrypoint /usr/bin/tar --security-opt label=disable -v /home/user/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.27.3-containerd-overlay2-amd64.tar.lz4:/preloaded.tar:ro -v minikube:/extractDir gcr.io/k8s-minikube/kicbase:v0.0.40 -I lz4 -xf /preloaded.tar -C /extractDir: (11.191727796s) I0720 17:52:39.362923 156176 kic.go:199] duration metric: took 11.191845 seconds to extract preloaded images to volume W0720 17:52:39.363205 156176 cgroups_linux.go:77] Your kernel does not support swap limit capabilities or the cgroup is not mounted. W0720 17:52:39.363268 156176 oci.go:240] Your kernel does not support CPU cfs period/quota or the cgroup is not mounted. I0720 17:52:39.363308 156176 cli_runner.go:164] Run: podman info --format "'{{json .SecurityOptions}}'" W0720 17:52:42.924179 156176 cli_runner.go:211] podman info --format "'{{json .SecurityOptions}}'" returned with exit code 125 I0720 17:52:42.924200 156176 cli_runner.go:217] Completed: podman info --format "'{{json .SecurityOptions}}'": (3.560838029s) I0720 17:52:42.924300 156176 cli_runner.go:164] Run: podman run -d -t --privileged --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname minikube --name minikube --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=minikube --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=minikube --network minikube --ip 192.168.49.2 --volume minikube:/var:exec --memory=7900mb -e container=podman --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 --publish=127.0.0.1::32443 gcr.io/k8s-minikube/kicbase:v0.0.40 W0720 17:52:49.242697 156176 cli_runner.go:211] podman run -d -t --privileged --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname minikube --name minikube --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=minikube --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=minikube --network minikube --ip 192.168.49.2 --volume minikube:/var:exec --memory=7900mb -e container=podman --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 --publish=127.0.0.1::32443 gcr.io/k8s-minikube/kicbase:v0.0.40 returned with exit code 126 I0720 17:52:49.242721 156176 cli_runner.go:217] Completed: podman run -d -t --privileged --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname minikube --name minikube --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=minikube --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=minikube --network minikube --ip 192.168.49.2 --volume minikube:/var:exec --memory=7900mb -e container=podman --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 --publish=127.0.0.1::32443 gcr.io/k8s-minikube/kicbase:v0.0.40: (6.318363385s) I0720 17:52:49.242772 156176 client.go:171] LocalClient.Create took 24.745587099s I0720 17:52:51.243476 156176 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'" I0720 17:52:51.243597 156176 cli_runner.go:164] Run: podman version --format {{.Version}} I0720 17:52:51.381367 156176 cli_runner.go:164] Run: podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube I0720 17:52:51.490301 156176 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:38079 SSHKeyPath:/home/user/.minikube/machines/minikube/id_rsa Username:docker} I0720 17:52:51.490533 156176 retry.go:31] will retry after 331.520036ms: new client: new client: Error creating new native config from ssh using: docker, &{[] [/home/user/.minikube/machines/minikube/id_rsa]}: open /home/user/.minikube/machines/minikube/id_rsa: no such file or directory I0720 17:52:51.823338 156176 cli_runner.go:164] Run: podman version --format {{.Version}} I0720 17:52:51.933714 156176 cli_runner.go:164] Run: podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube I0720 17:52:52.052250 156176 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:38079 SSHKeyPath:/home/user/.minikube/machines/minikube/id_rsa Username:docker} I0720 17:52:52.052448 156176 retry.go:31] will retry after 236.672894ms: new client: new client: Error creating new native config from ssh using: docker, &{[] [/home/user/.minikube/machines/minikube/id_rsa]}: open /home/user/.minikube/machines/minikube/id_rsa: no such file or directory I0720 17:52:52.290302 156176 cli_runner.go:164] Run: podman version --format {{.Version}} I0720 17:52:52.396626 156176 cli_runner.go:164] Run: podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube I0720 17:52:52.448771 156176 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:38079 SSHKeyPath:/home/user/.minikube/machines/minikube/id_rsa Username:docker} I0720 17:52:52.448864 156176 retry.go:31] will retry after 686.150186ms: new client: new client: Error creating new native config from ssh using: docker, &{[] [/home/user/.minikube/machines/minikube/id_rsa]}: open /home/user/.minikube/machines/minikube/id_rsa: no such file or directory I0720 17:52:53.135305 156176 cli_runner.go:164] Run: podman version --format {{.Version}} I0720 17:52:53.253209 156176 cli_runner.go:164] Run: podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube I0720 17:52:53.371803 156176 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:38079 SSHKeyPath:/home/user/.minikube/machines/minikube/id_rsa Username:docker} W0720 17:52:53.372134 156176 start.go:275] error running df -h /var: NewSession: new client: new client: Error creating new native config from ssh using: docker, &{[] [/home/user/.minikube/machines/minikube/id_rsa]}: open /home/user/.minikube/machines/minikube/id_rsa: no such file or directory W0720 17:52:53.372198 156176 start.go:242] error getting percentage of /var that is free: NewSession: new client: new client: Error creating new native config from ssh using: docker, &{[] [/home/user/.minikube/machines/minikube/id_rsa]}: open /home/user/.minikube/machines/minikube/id_rsa: no such file or directory I0720 17:52:53.372312 156176 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'" I0720 17:52:53.372400 156176 cli_runner.go:164] Run: podman version --format {{.Version}} I0720 17:52:53.489449 156176 cli_runner.go:164] Run: podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube I0720 17:52:53.599127 156176 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:38079 SSHKeyPath:/home/user/.minikube/machines/minikube/id_rsa Username:docker} I0720 17:52:53.599215 156176 retry.go:31] will retry after 368.372712ms: new client: new client: Error creating new native config from ssh using: docker, &{[] [/home/user/.minikube/machines/minikube/id_rsa]}: open /home/user/.minikube/machines/minikube/id_rsa: no such file or directory I0720 17:52:53.968426 156176 cli_runner.go:164] Run: podman version --format {{.Version}} I0720 17:52:54.055536 156176 cli_runner.go:164] Run: podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube I0720 17:52:54.096379 156176 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:38079 SSHKeyPath:/home/user/.minikube/machines/minikube/id_rsa Username:docker} I0720 17:52:54.096463 156176 retry.go:31] will retry after 351.643475ms: new client: new client: Error creating new native config from ssh using: docker, &{[] [/home/user/.minikube/machines/minikube/id_rsa]}: open /home/user/.minikube/machines/minikube/id_rsa: no such file or directory I0720 17:52:54.448719 156176 cli_runner.go:164] Run: podman version --format {{.Version}} I0720 17:52:54.554354 156176 cli_runner.go:164] Run: podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube I0720 17:52:54.655829 156176 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:38079 SSHKeyPath:/home/user/.minikube/machines/minikube/id_rsa Username:docker} I0720 17:52:54.655894 156176 retry.go:31] will retry after 621.330498ms: new client: new client: Error creating new native config from ssh using: docker, &{[] [/home/user/.minikube/machines/minikube/id_rsa]}: open /home/user/.minikube/machines/minikube/id_rsa: no such file or directory I0720 17:52:55.278000 156176 cli_runner.go:164] Run: podman version --format {{.Version}} I0720 17:52:55.413040 156176 cli_runner.go:164] Run: podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube I0720 17:52:55.538557 156176 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:38079 SSHKeyPath:/home/user/.minikube/machines/minikube/id_rsa Username:docker} W0720 17:52:55.538931 156176 start.go:290] error running df -BG /var: NewSession: new client: new client: Error creating new native config from ssh using: docker, &{[] [/home/user/.minikube/machines/minikube/id_rsa]}: open /home/user/.minikube/machines/minikube/id_rsa: no such file or directory W0720 17:52:55.538991 156176 start.go:247] error getting GiB of /var that is available: NewSession: new client: new client: Error creating new native config from ssh using: docker, &{[] [/home/user/.minikube/machines/minikube/id_rsa]}: open /home/user/.minikube/machines/minikube/id_rsa: no such file or directory I0720 17:52:55.539062 156176 start.go:128] duration metric: createHost completed in 31.046274902s I0720 17:52:55.539082 156176 start.go:83] releasing machines lock for "minikube", held for 31.046412855s W0720 17:52:55.539124 156176 start.go:672] error starting host: creating host: create: creating: create kic node: create container: podman run -d -t --privileged --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname minikube --name minikube --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=minikube --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=minikube --network minikube --ip 192.168.49.2 --volume minikube:/var:exec --memory=7900mb -e container=podman --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 --publish=127.0.0.1::32443 gcr.io/k8s-minikube/kicbase:v0.0.40: exit status 126 stdout:
stderr: Error: crun: error stat'ing file
/dev/vboxusb/001/003: Permission denied: OCI permission denied I0720 17:52:55.541767 156176 cli_runner.go:164] Run: podman container inspect minikube --format={{.State.Status}} W0720 17:52:55.648448 156176 start.go:677] delete host: Docker machine "minikube" does not exist. Use "docker-machine ls" to list machines. Use "docker-machine create" to add a new one. W0720 17:52:55.648518 156176 out.go:239] ๐คฆ StartHost failed, but will try again: creating host: create: creating: create kic node: create container: podman run -d -t --privileged --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname minikube --name minikube --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=minikube --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=minikube --network minikube --ip 192.168.49.2 --volume minikube:/var:exec --memory=7900mb -e container=podman --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 --publish=127.0.0.1::32443 gcr.io/k8s-minikube/kicbase:v0.0.40: exit status 126 stdout:stderr: Error: crun: error stat'ing file
/dev/vboxusb/001/003: Permission denied: OCI permission deniedI0720 17:52:55.648588 156176 start.go:687] Will try again in 5 seconds ... I0720 17:53:00.651194 156176 start.go:365] acquiring machines lock for minikube: {Name:mk014e19cf8b44f5e25987f528a80e8a38617d4e Clock:{} Delay:500ms Timeout:10m0s Cancel:}
I0720 17:53:00.651342 156176 start.go:369] acquired machines lock for "minikube" in 109.031ยตs
I0720 17:53:00.651378 156176 start.go:96] Skipping create...Using existing machine configuration
I0720 17:53:00.651387 156176 fix.go:54] fixHost starting:
I0720 17:53:00.651961 156176 cli_runner.go:164] Run: podman container inspect minikube --format={{.State.Status}}
I0720 17:53:00.750839 156176 fix.go:102] recreateIfNeeded on minikube: state= err=
W0720 17:53:00.750854 156176 fix.go:128] unexpected machine state, will restart:
I0720 17:53:00.762916 156176 out.go:177] ๐ Restarting existing podman container for "minikube" ...
I0720 17:53:00.766778 156176 cli_runner.go:164] Run: podman start minikube
W0720 17:53:01.501354 156176 cli_runner.go:211] podman start minikube returned with exit code 125
I0720 17:53:01.501397 156176 cli_runner.go:164] Run: podman inspect minikube
I0720 17:53:01.574370 156176 errors.go:84] Postmortem inspect ("podman inspect minikube"): -- stdout --
[
{
"Id": "606e6a197505aceb29e4ececd6ebf73942619ce9559ebc04a507a00e3db8b4a2",
"Created": "2023-07-20T17:52:43.060528479+02:00",
"Path": "/usr/local/bin/entrypoint",
"Args": [
"/sbin/init"
],
"State": {
"OciVersion": "1.0.2-dev",
"Status": "created",
"Running": false,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 0,
"ExitCode": 0,
"Error": "",
"StartedAt": "0001-01-01T00:00:00Z",
"FinishedAt": "0001-01-01T00:00:00Z",
"Health": {
"Status": "",
"FailingStreak": 0,
"Log": null
},
"CheckpointedAt": "0001-01-01T00:00:00Z",
"RestoredAt": "0001-01-01T00:00:00Z"
},
"Image": "c6cc01e6091959400f260dc442708e7c71630b58dab1f7c344cb00926bd84950",
"ImageDigest": "sha256:8cadf23777709e43eca447c47a45f5a4635615129267ce025193040ec92a1631",
"ImageName": "gcr.io/k8s-minikube/kicbase:v0.0.40",
"Rootfs": "",
"Pod": "",
"ResolvConfPath": "/run/user/1000/containers/vfs-containers/606e6a197505aceb29e4ececd6ebf73942619ce9559ebc04a507a00e3db8b4a2/userdata/resolv.conf",
"HostnamePath": "/run/user/1000/containers/vfs-containers/606e6a197505aceb29e4ececd6ebf73942619ce9559ebc04a507a00e3db8b4a2/userdata/hostname",
"HostsPath": "/run/user/1000/containers/vfs-containers/606e6a197505aceb29e4ececd6ebf73942619ce9559ebc04a507a00e3db8b4a2/userdata/hosts",
"StaticDir": "/home/user/.local/share/containers/storage/vfs-containers/606e6a197505aceb29e4ececd6ebf73942619ce9559ebc04a507a00e3db8b4a2/userdata",
"OCIConfigPath": "/home/user/.local/share/containers/storage/vfs-containers/606e6a197505aceb29e4ececd6ebf73942619ce9559ebc04a507a00e3db8b4a2/userdata/config.json",
"OCIRuntime": "crun",
"ConmonPidFile": "/run/user/1000/containers/vfs-containers/606e6a197505aceb29e4ececd6ebf73942619ce9559ebc04a507a00e3db8b4a2/userdata/conmon.pid",
"PidFile": "/run/user/1000/containers/vfs-containers/606e6a197505aceb29e4ececd6ebf73942619ce9559ebc04a507a00e3db8b4a2/userdata/pidfile",
"Name": "minikube",
"RestartCount": 0,
"Driver": "vfs",
"MountLabel": "",
"ProcessLabel": "",
"AppArmorProfile": "",
"EffectiveCaps": [
"CAP_AUDIT_CONTROL",
"CAP_AUDIT_READ",
"CAP_AUDIT_WRITE",
"CAP_BLOCK_SUSPEND",
"CAP_BPF",
"CAP_CHECKPOINT_RESTORE",
"CAP_CHOWN",
"CAP_DAC_OVERRIDE",
"CAP_DAC_READ_SEARCH",
"CAP_FOWNER",
"CAP_FSETID",
"CAP_IPC_LOCK",
"CAP_IPC_OWNER",
"CAP_KILL",
"CAP_LEASE",
"CAP_LINUX_IMMUTABLE",
"CAP_MAC_ADMIN",
"CAP_MAC_OVERRIDE",
"CAP_MKNOD",
"CAP_NET_ADMIN",
"CAP_NET_BIND_SERVICE",
"CAP_NET_BROADCAST",
"CAP_NET_RAW",
"CAP_PERFMON",
"CAP_SETFCAP",
"CAP_SETGID",
"CAP_SETPCAP",
"CAP_SETUID",
"CAP_SYSLOG",
"CAP_SYS_ADMIN",
"CAP_SYS_BOOT",
"CAP_SYS_CHROOT",
"CAP_SYS_MODULE",
"CAP_SYS_NICE",
"CAP_SYS_PACCT",
"CAP_SYS_PTRACE",
"CAP_SYS_RAWIO",
"CAP_SYS_RESOURCE",
"CAP_SYS_TIME",
"CAP_SYS_TTY_CONFIG",
"CAP_WAKE_ALARM"
],
"BoundingCaps": [
"CAP_AUDIT_CONTROL",
"CAP_AUDIT_READ",
"CAP_AUDIT_WRITE",
"CAP_BLOCK_SUSPEND",
"CAP_BPF",
"CAP_CHECKPOINT_RESTORE",
"CAP_CHOWN",
"CAP_DAC_OVERRIDE",
"CAP_DAC_READ_SEARCH",
"CAP_FOWNER",
"CAP_FSETID",
"CAP_IPC_LOCK",
"CAP_IPC_OWNER",
"CAP_KILL",
"CAP_LEASE",
"CAP_LINUX_IMMUTABLE",
"CAP_MAC_ADMIN",
"CAP_MAC_OVERRIDE",
"CAP_MKNOD",
"CAP_NET_ADMIN",
"CAP_NET_BIND_SERVICE",
"CAP_NET_BROADCAST",
"CAP_NET_RAW",
"CAP_PERFMON",
"CAP_SETFCAP",
"CAP_SETGID",
"CAP_SETPCAP",
"CAP_SETUID",
"CAP_SYSLOG",
"CAP_SYS_ADMIN",
"CAP_SYS_BOOT",
"CAP_SYS_CHROOT",
"CAP_SYS_MODULE",
"CAP_SYS_NICE",
"CAP_SYS_PACCT",
"CAP_SYS_PTRACE",
"CAP_SYS_RAWIO",
"CAP_SYS_RESOURCE",
"CAP_SYS_TIME",
"CAP_SYS_TTY_CONFIG",
"CAP_WAKE_ALARM"
],
"ExecIDs": [],
"GraphDriver": {
"Name": "vfs",
"Data": null
},
"Mounts": [
{
"Type": "volume",
"Name": "minikube",
"Source": "/home/user/.local/share/containers/storage/volumes/minikube/_data",
"Destination": "/var",
"Driver": "local",
"Mode": "",
"Options": [
"exec",
"nosuid",
"nodev",
"rbind"
],
"RW": true,
"Propagation": "rprivate"
},
{
"Type": "bind",
"Source": "/lib/modules",
"Destination": "/lib/modules",
"Driver": "",
"Mode": "",
"Options": [
"rbind"
],
"RW": false,
"Propagation": "rprivate"
}
],
"Dependencies": [],
"NetworkSettings": {
"EndpointID": "",
"Gateway": "",
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"MacAddress": "",
"Bridge": "",
"SandboxID": "",
"HairpinMode": false,
"LinkLocalIPv6Address": "",
"LinkLocalIPv6PrefixLen": 0,
"Ports": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "38079"
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "41757"
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "37569"
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "41015"
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "36917"
}
]
},
"SandboxKey": "",
"Networks": {
"minikube": {
"EndpointID": "",
"Gateway": "",
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"MacAddress": "",
"NetworkID": "minikube",
"DriverOpts": null,
"IPAMConfig": null,
"Links": null,
"Aliases": [
"606e6a197505"
]
}
}
},
"Namespace": "",
"IsInfra": false,
"IsService": false,
"Config": {
"Hostname": "minikube",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"Tty": true,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"container=podman",
"HTTP_PROXY=http://internal.local:3130/",
"http_proxy=http://internal.local:3130/",
"HTTPS_PROXY=http://internal.local:3130/",
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
"ftp_proxy=http://internal.local:3130/",
"FTP_PROXY=http://internal.local:3130/",
"https_proxy=http://internal.local:3130/",
"TERM=xterm",
"HOSTNAME=minikube",
"HOME=/root"
],
"Cmd": null,
"Image": "gcr.io/k8s-minikube/kicbase:v0.0.40",
"Volumes": null,
"WorkingDir": "/",
"Entrypoint": "/usr/local/bin/entrypoint /sbin/init",
"OnBuild": null,
"Labels": {
"created_by.minikube.sigs.k8s.io": "true",
"mode.minikube.sigs.k8s.io": "minikube",
"name.minikube.sigs.k8s.io": "minikube",
"role.minikube.sigs.k8s.io": ""
},
"Annotations": {
"io.container.manager": "libpod",
"io.kubernetes.cri-o.Created": "2023-07-20T17:52:43.060528479+02:00",
"io.kubernetes.cri-o.TTY": "true",
"io.podman.annotations.autoremove": "FALSE",
"io.podman.annotations.init": "FALSE",
"io.podman.annotations.privileged": "TRUE",
"io.podman.annotations.publish-all": "FALSE",
"io.podman.annotations.seccomp": "unconfined",
"org.opencontainers.image.stopSignal": "37"
},
"StopSignal": 37,
"HealthcheckOnFailureAction": "none",
"CreateCommand": [
"podman",
"run",
"-d",
"-t",
"--privileged",
"--security-opt",
"seccomp=unconfined",
"--tmpfs",
"/tmp",
"--tmpfs",
"/run",
"-v",
"/lib/modules:/lib/modules:ro",
"--hostname",
"minikube",
"--name",
"minikube",
"--label",
"created_by.minikube.sigs.k8s.io=true",
"--label",
"name.minikube.sigs.k8s.io=minikube",
"--label",
"role.minikube.sigs.k8s.io=",
"--label",
"mode.minikube.sigs.k8s.io=minikube",
"--network",
"minikube",
"--ip",
"192.168.49.2",
"--volume",
"minikube:/var:exec",
"--memory=7900mb",
"-e",
"container=podman",
"--expose",
"8443",
"--publish=127.0.0.1::8443",
"--publish=127.0.0.1::22",
"--publish=127.0.0.1::2376",
"--publish=127.0.0.1::5000",
"--publish=127.0.0.1::32443",
"gcr.io/k8s-minikube/kicbase:v0.0.40"
],
"Umask": "0022",
"Timeout": 0,
"StopTimeout": 10,
"Passwd": true,
"sdNotifyMode": "container"
},
"HostConfig": {
"Binds": [
"minikube:/var:exec,rw,rprivate,nosuid,nodev,rbind",
"/lib/modules:/lib/modules:ro,rprivate,rbind"
],
"CgroupManager": "systemd",
"CgroupMode": "private",
"ContainerIDFile": "",
"LogConfig": {
"Type": "journald",
"Config": null,
"Path": "",
"Tag": "",
"Size": "0B"
},
"NetworkMode": "bridge",
"PortBindings": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "38079"
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "41757"
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "37569"
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "41015"
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "36917"
}
]
},
"RestartPolicy": {
"Name": "",
"MaximumRetryCount": 0
},
"AutoRemove": false,
"VolumeDriver": "",
"VolumesFrom": null,
"CapAdd": [],
"CapDrop": [],
"Dns": [],
"DnsOptions": [],
"DnsSearch": [],
"ExtraHosts": [],
"GroupAdd": [],
"IpcMode": "shareable",
"Cgroup": "",
"Cgroups": "default",
"Links": null,
"OomScoreAdj": 0,
"PidMode": "private",
"Privileged": true,
"PublishAllPorts": false,
"ReadonlyRootfs": false,
"SecurityOpt": [
"seccomp=unconfined"
],
"Tmpfs": {
"/run": "rw,rprivate,nosuid,nodev,tmpcopyup",
"/tmp": "rw,rprivate,nosuid,nodev,tmpcopyup"
},
"UTSMode": "private",
"UsernsMode": "",
"ShmSize": 65536000,
"Runtime": "oci",
"ConsoleSize": [
0,
0
],
"Isolation": "",
"CpuShares": 0,
"Memory": 8283750400,
"NanoCpus": 0,
"CgroupParent": "user.slice",
"BlkioWeight": 0,
"BlkioWeightDevice": null,
"BlkioDeviceReadBps": null,
"BlkioDeviceWriteBps": null,
"BlkioDeviceReadIOps": null,
"BlkioDeviceWriteIOps": null,
"CpuPeriod": 0,
"CpuQuota": 0,
"CpuRealtimePeriod": 0,
"CpuRealtimeRuntime": 0,
"CpusetCpus": "",
"CpusetMems": "",
"Devices": [],
"DiskQuota": 0,
"KernelMemory": 0,
"MemoryReservation": 0,
"MemorySwap": 16567500800,
"MemorySwappiness": -1,
"OomKillDisable": false,
"PidsLimit": 2048,
"Ulimits": [],
"CpuCount": 0,
"CpuPercent": 0,
"IOMaximumIOps": 0,
"IOMaximumBandwidth": 0,
"CgroupConf": null
}
}
]
-- /stdout -- I0720 17:53:01.574423 156176 cli_runner.go:164] Run: podman logs --timestamps minikube I0720 17:53:01.725014 156176 errors.go:91] Postmortem logs ("podman logs --timestamps minikube"): I0720 17:53:01.725089 156176 cli_runner.go:164] Run: podman system info --format json I0720 17:53:04.953456 156176 cli_runner.go:217] Completed: podman system info --format json: (3.22834532s) I0720 17:53:04.953567 156176 info.go:288] podman info: {Host:{BuildahVersion:1.28.2 CgroupVersion:v2 Conmon:{Package:conmon_2.1.6+ds1-1_amd64 Path:/usr/bin/conmon Version:conmon version 2.1.6, commit: unknown} Distribution:{Distribution:debian Version:12} MemFree:699437056 MemTotal:33273450496 OCIRuntime:{Name:crun Package:crun_1.8.1-1+b1_amd64 Path:/usr/bin/crun Version:crun version 1.8.1 commit: f8a096be060b22ccd3d5f3ebe44108517fbf6c30 rundir: /run/user/1000/crun spec: 1.0.0 +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL} SwapFree:16575623168 SwapTotal:16651382784 Arch:amd64 Cpus:8 Eventlogger:journald Hostname:FRSIEL11142 Kernel:6.1.0-9-amd64 Os:linux Security:{Rootless:true} Uptime:7h 59m 49.00s (Approximately 0.29 days)} Registries:{Search:[]} Store:{ConfigFile:/home/user/.config/containers/storage.conf ContainerStore:{Number:1} GraphDriverName:vfs GraphOptions:{} GraphRoot:/home/user/.local/share/containers/storage GraphStatus:{BackingFilesystem: NativeOverlayDiff: SupportsDType: UsingMetacopy:} ImageStore:{Number:1} RunRoot:/run/user/1000/containers VolumePath:/home/user/.local/share/containers/storage/volumes}} I0720 17:53:04.953591 156176 errors.go:106] postmortem podman info: {Host:{BuildahVersion:1.28.2 CgroupVersion:v2 Conmon:{Package:conmon_2.1.6+ds1-1_amd64 Path:/usr/bin/conmon Version:conmon version 2.1.6, commit: unknown} Distribution:{Distribution:debian Version:12} MemFree:699437056 MemTotal:33273450496 OCIRuntime:{Name:crun Package:crun_1.8.1-1+b1_amd64 Path:/usr/bin/crun Version:crun version 1.8.1 commit: f8a096be060b22ccd3d5f3ebe44108517fbf6c30 rundir: /run/user/1000/crun spec: 1.0.0 +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL} SwapFree:16575623168 SwapTotal:16651382784 Arch:amd64 Cpus:8 Eventlogger:journald Hostname:FRSIEL11142 Kernel:6.1.0-9-amd64 Os:linux Security:{Rootless:true} Uptime:7h 59m 49.00s (Approximately 0.29 days)} Registries:{Search:[]} Store:{ConfigFile:/home/user/.config/containers/storage.conf ContainerStore:{Number:1} GraphDriverName:vfs GraphOptions:{} GraphRoot:/home/user/.local/share/containers/storage GraphStatus:{BackingFilesystem: NativeOverlayDiff: SupportsDType: UsingMetacopy:} ImageStore:{Number:1} RunRoot:/run/user/1000/containers VolumePath:/home/user/.local/share/containers/storage/volumes}} I0720 17:53:04.953626 156176 network_create.go:281] running [podman network inspect minikube] to gather additional debugging logs... I0720 17:53:04.953640 156176 cli_runner.go:164] Run: podman network inspect minikube I0720 17:53:05.015723 156176 network_create.go:286] output of [podman network inspect minikube]: -- stdout -- [ { "name": "minikube", "id": "7a38d9133e293225d3e21fa37439d5062a3399221571bcb7ddf30f0a80cc5d75", "driver": "bridge", "network_interface": "podman1", "created": "2023-07-20T17:52:24.761261983+02:00", "subnets": [ { "subnet": "192.168.49.0/24", "gateway": "192.168.49.1" } ], "ipv6_enabled": false, "internal": false, "dns_enabled": true, "labels": { "created_by.minikube.sigs.k8s.io": "true", "name.minikube.sigs.k8s.io": "minikube" }, "ipam_options": { "driver": "host-local" } } ]
-- /stdout -- I0720 17:53:05.015778 156176 cli_runner.go:164] Run: podman system info --format json I0720 17:53:08.215273 156176 cli_runner.go:217] Completed: podman system info --format json: (3.199474444s) I0720 17:53:08.215377 156176 info.go:288] podman info: {Host:{BuildahVersion:1.28.2 CgroupVersion:v2 Conmon:{Package:conmon_2.1.6+ds1-1_amd64 Path:/usr/bin/conmon Version:conmon version 2.1.6, commit: unknown} Distribution:{Distribution:debian Version:12} MemFree:865320960 MemTotal:33273450496 OCIRuntime:{Name:crun Package:crun_1.8.1-1+b1_amd64 Path:/usr/bin/crun Version:crun version 1.8.1 commit: f8a096be060b22ccd3d5f3ebe44108517fbf6c30 rundir: /run/user/1000/crun spec: 1.0.0 +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL} SwapFree:16575623168 SwapTotal:16651382784 Arch:amd64 Cpus:8 Eventlogger:journald Hostname:FRSIEL11142 Kernel:6.1.0-9-amd64 Os:linux Security:{Rootless:true} Uptime:7h 59m 52.00s (Approximately 0.29 days)} Registries:{Search:[]} Store:{ConfigFile:/home/user/.config/containers/storage.conf ContainerStore:{Number:1} GraphDriverName:vfs GraphOptions:{} GraphRoot:/home/user/.local/share/containers/storage GraphStatus:{BackingFilesystem: NativeOverlayDiff: SupportsDType: UsingMetacopy:} ImageStore:{Number:1} RunRoot:/run/user/1000/containers VolumePath:/home/user/.local/share/containers/storage/volumes}} I0720 17:53:08.215457 156176 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'" I0720 17:53:08.215482 156176 cli_runner.go:164] Run: podman version --format {{.Version}} I0720 17:53:08.298213 156176 cli_runner.go:164] Run: podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube I0720 17:53:08.420061 156176 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:38079 SSHKeyPath:/home/user/.minikube/machines/minikube/id_rsa Username:docker} I0720 17:53:08.420276 156176 retry.go:31] will retry after 225.530645ms: new client: new client: Error creating new native config from ssh using: docker, &{[] [/home/user/.minikube/machines/minikube/id_rsa]}: open /home/user/.minikube/machines/minikube/id_rsa: no such file or directory I0720 17:53:08.646827 156176 cli_runner.go:164] Run: podman version --format {{.Version}} I0720 17:53:08.724052 156176 cli_runner.go:164] Run: podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube I0720 17:53:08.818672 156176 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:38079 SSHKeyPath:/home/user/.minikube/machines/minikube/id_rsa Username:docker} I0720 17:53:08.818775 156176 retry.go:31] will retry after 376.992331ms: new client: new client: Error creating new native config from ssh using: docker, &{[] [/home/user/.minikube/machines/minikube/id_rsa]}: open /home/user/.minikube/machines/minikube/id_rsa: no such file or directory I0720 17:53:09.196651 156176 cli_runner.go:164] Run: podman version --format {{.Version}} I0720 17:53:09.299874 156176 cli_runner.go:164] Run: podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube I0720 17:53:09.344280 156176 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:38079 SSHKeyPath:/home/user/.minikube/machines/minikube/id_rsa Username:docker} I0720 17:53:09.344347 156176 retry.go:31] will retry after 456.954268ms: new client: new client: Error creating new native config from ssh using: docker, &{[] [/home/user/.minikube/machines/minikube/id_rsa]}: open /home/user/.minikube/machines/minikube/id_rsa: no such file or directory I0720 17:53:09.802200 156176 cli_runner.go:164] Run: podman version --format {{.Version}} I0720 17:53:09.926292 156176 cli_runner.go:164] Run: podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube I0720 17:53:10.026608 156176 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:38079 SSHKeyPath:/home/user/.minikube/machines/minikube/id_rsa Username:docker} W0720 17:53:10.026822 156176 start.go:275] error running df -h /var: NewSession: new client: new client: Error creating new native config from ssh using: docker, &{[] [/home/user/.minikube/machines/minikube/id_rsa]}: open /home/user/.minikube/machines/minikube/id_rsa: no such file or directory W0720 17:53:10.026848 156176 start.go:242] error getting percentage of /var that is free: NewSession: new client: new client: Error creating new native config from ssh using: docker, &{[] [/home/user/.minikube/machines/minikube/id_rsa]}: open /home/user/.minikube/machines/minikube/id_rsa: no such file or directory I0720 17:53:10.026911 156176 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'" I0720 17:53:10.026969 156176 cli_runner.go:164] Run: podman version --format {{.Version}} I0720 17:53:10.118862 156176 cli_runner.go:164] Run: podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube I0720 17:53:10.208754 156176 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:38079 SSHKeyPath:/home/user/.minikube/machines/minikube/id_rsa Username:docker} I0720 17:53:10.208814 156176 retry.go:31] will retry after 281.478285ms: new client: new client: Error creating new native config from ssh using: docker, &{[] [/home/user/.minikube/machines/minikube/id_rsa]}: open /home/user/.minikube/machines/minikube/id_rsa: no such file or directory I0720 17:53:10.491460 156176 cli_runner.go:164] Run: podman version --format {{.Version}} I0720 17:53:10.612094 156176 cli_runner.go:164] Run: podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube I0720 17:53:10.698805 156176 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:38079 SSHKeyPath:/home/user/.minikube/machines/minikube/id_rsa Username:docker} I0720 17:53:10.698878 156176 retry.go:31] will retry after 256.111449ms: new client: new client: Error creating new native config from ssh using: docker, &{[] [/home/user/.minikube/machines/minikube/id_rsa]}: open /home/user/.minikube/machines/minikube/id_rsa: no such file or directory I0720 17:53:10.955507 156176 cli_runner.go:164] Run: podman version --format {{.Version}} I0720 17:53:11.099117 156176 cli_runner.go:164] Run: podman container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube I0720 17:53:11.237857 156176 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:38079 SSHKeyPath:/home/user/.minikube/machines/minikube/id_rsa Username:docker} W0720 17:53:11.238028 156176 start.go:290] error running df -BG /var: NewSession: new client: new client: Error creating new native config from ssh using: docker, &{[] [/home/user/.minikube/machines/minikube/id_rsa]}: open /home/user/.minikube/machines/minikube/id_rsa: no such file or directory W0720 17:53:11.238057 156176 start.go:247] error getting GiB of /var that is available: NewSession: new client: new client: Error creating new native config from ssh using: docker, &{[] [/home/user/.minikube/machines/minikube/id_rsa]}: open /home/user/.minikube/machines/minikube/id_rsa: no such file or directory I0720 17:53:11.238071 156176 fix.go:56] fixHost completed within 10.586684215s I0720 17:53:11.238082 156176 start.go:83] releasing machines lock for "minikube", held for 10.586726087s W0720 17:53:11.238197 156176 out.go:239] ๐ฟ Failed to start podman container. Running "minikube delete" may fix it: driver start: start: podman start minikube: exit status 125 stdout:
stderr: Error: unable to start container "606e6a197505aceb29e4ececd6ebf73942619ce9559ebc04a507a00e3db8b4a2": crun: error stat'ing file
/dev/vboxusb/001/003: Permission denied: OCI permission deniedI0720 17:53:11.245120 156176 out.go:177] W0720 17:53:11.248844 156176 out.go:239] โ Exiting due to GUEST_PROVISION: error provisioning guest: Failed to start host: driver start: start: podman start minikube: exit status 125 stdout:
stderr: Error: unable to start container "606e6a197505aceb29e4ececd6ebf73942619ce9559ebc04a507a00e3db8b4a2": crun: error stat'ing file
/dev/vboxusb/001/003: Permission denied: OCI permission denied `Operating System
Other
Driver
Podman