Question: does skaffold support Minikube with podman?

stephanwesten commented 7 months ago

Hi,

Since my company moved away from Docker (Desktop), we have settled on Podman. My question: Is Podman supported?

Reading the documentation like https://skaffold.dev/docs/builders/builder-types/docker/#dockerfile-with-docker-locally I get the impression that I need to have Docker.

I also tried running Minikube with 'minikube start --driver=podman --container-runtime=cri-o', but when i use skaffold dev commands, I get errors like "the docker-env command only supports the docker and containerd runtimes"

So it seems you need to have Docker running. However the https://skaffold.dev/docs/quickstart/ does not list Docker. Hence my confusion.

ericzzzzzzz commented 7 months ago

Hi, @stephanwesten , to use podman you need to do two things

~set DOCKER_HOST env variable to your podman socket, please take a look this to see how to do it on different platform, https://podman-desktop.io/docs/migrating-from-docker/using-the-docker_host-environment-variable~ (not required if using minikube)
pass --check-cluster-node-platforms=false flag when running skaffold command , the flag won't be necessary on the next release for using podman : ) , however, this should only work on linux system : (

stephanwesten commented 7 months ago

Ah, I'm using a Mac....

So I should conclude that skaffold does not support Podman on Mac?

Also in the next release it won't work? (that leaves out quite an audience...)

ericzzzzzzz commented 7 months ago

Ah, I'm using a Mac....

So I should conclude that skaffold does not support Podman on Mac?

Also in the next release it won't work? (that leaves out quite an audience...)

sorry for the confusion , You can use --check-cluster-node-platforms=false for any OS. However, to make podman work without that flag, that requires a bit extra work, without flag it will only work on linux for the next release.

stephanwesten commented 7 months ago

Unfortunately I don't even make it to the skaffold command...

Although you mentioned not needed, I did set the DOCKER_HOST:

➜ skaffoldtest echo $DOCKER_HOST
unix:///Users/stephan.westen/.local/share/containers/podman/machine/qemu/podman.sock

I get the an error when starting Minikube with containerd - due to the driver apparently:

➜ skaffoldtest minikube start --driver=podman --container-runtime=containerd --profile custom 😄 [custom] minikube v1.32.0 on Darwin 13.6.2 (arm64) ✨ Using the podman (experimental) driver based on existing profile

⛔ Requested memory allocation (1888MB) is less than the recommended minimum 1900MB. Deployments may fail.

👍 Starting control plane node custom in cluster custom 🚜 Pulling base image ... 💾 Downloading Kubernetes v1.28.3 preload ...

preloaded-images-k8s-v18-v1...: 371.07 MiB / 371.07 MiB 100.00% 28.06 M E1129 11:00:12.308132 3492 cache.go:189] Error downloading kic artifacts: not yet implemented, see issue #8426 🏃 Updating the running podman "custom" container ... 📦 Preparing Kubernetes v1.28.3 on containerd 1.6.24 ... 🔗 Configuring CNI (Container Networking Interface) ... 🔎 Verifying Kubernetes components... ▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5 🌟 Enabled addons: storage-provisioner, default-storageclass 🏄 Done! kubectl is now configured to use "custom" cluster and "default" namespace by default ➜ skaffoldtest eval $(minikube -p custom docker-env) ❌ Exiting due to MK_USAGE: the docker-env command only supports the containerd runtime with the docker driver

➜ skaffoldtest skaffold dev --check-cluster-node-platforms=false
invalid skaffold config: getting minikube env: running [/usr/local/bin/minikube docker-env --shell none -p custom --user=skaffold]

stdout: "false exit code 14\n"
stderr: "X Exiting due to MK_USAGE: the docker-env command only supports the containerd runtime with the docker driver\n"
cause: exit status 14

ericzzzzzzz commented 7 months ago

yeah. Skaffold uses minikube dokcer-env to get builder environment variables, that command only works with when minikube cluster is configured with docker runtime , is it possible for you to use docker as container-runtime on minikube ?

stephanwesten commented 7 months ago

I could.

As more people will follow the same path as I tried, I suggest to update the docs.

Thanks for the help!

afbjorklund commented 6 months ago

Minikube supports multiple container runtimes, but skaffold is hardcoded to only use docker...

https://github.com/GoogleContainerTools/skaffold/issues/5739

Therefore, we need to add legacy compatibility layers to minikube for cri-o and for containerd.

https://github.com/kubernetes/minikube/issues/15389

GoogleContainerTools / skaffold

Question: does skaffold support Minikube with podman? #9194