GoogleContainerTools / skaffold

Easy and Repeatable Kubernetes Development
https://skaffold.dev/
Apache License 2.0
15k stars 1.62k forks source link

Spurious warning when using minikube with "none" driver #7711

Open gilbahat opened 2 years ago

gilbahat commented 2 years ago

Skaffold is confused when running under local "none" minikube.

~$ minikube docker-env

❌ Exiting due to ENV_DRIVER_CONFLICT: 'none' driver does not support 'minikube docker-env' command

Expected behavior

Skaffold ignores the specific error issued by minikube when the 'none' driver is used

Actual behavior

Skaffold complains about the output of minikube docker-env but proceeds correctly: WARN[0000] Could not get minikube docker env, falling back to local docker daemon: getting minikube env: running [/usr/local/bin/minikube docker-env --shell none -p minikube --user=skaffold]

note - this is kinda cosmetic but I wonder if it could be an issue with other nonstandard drivers such as ssh.

Information

Steps to reproduce the behavior

  1. install a none-driver minikube instance - https://minikube.sigs.k8s.io/docs/drivers/none/
  2. run any skaffold build
tejal29 commented 2 years ago

relates to https://github.com/GoogleContainerTools/skaffold/issues/4654

tejal29 commented 2 years ago

Background

Minikube can be deployed as a VM, a container, or bare-metal. In case of container, user can choose the docker driver. (list here)

When minikube is started with docker driver, the k8s cluster is installed into an existing Docker install.

Currently in skaffold, if we detect a user has minikube as their active kubecontext, we run a command minikube docker-env so that docker commands on the host machine build to minikube's docker daemon. What this means, running a docker buildwill run inside minikube's docker daemon and the image will be available on the minikube cluster without a need to push.

Here is why, it is beneficial to use docker's deamon. https://stackoverflow.com/questions/52310599/what-does-minikube-docker-env-mean/52310892#52310892

Problem:

docker driver flow was the most used flow before k8s decided to deprecate docker shim A lot of users started using different container runtimes or drivers when creating minikube cluster. Skaffold needs to 1) verify if the minikube is using docker driver. There is a command to do so on the minikube docker driver page. 2) If the driver is not other than docker, skaffold should use the minikube image load command which can load a image into the minikube cluster. https://minikube.sigs.k8s.io/docs/handbook/pushing/#7-loading-directly-to-in-cluster-container-runtime This will load the image from the local docker daemon into minikube cluster and k8s app will be pull this image.

Starting points:

1) look for minikube docker env code in getMinikubeDockerEnv. 2) Add a change similar to this LoadImages here to detect if the cluster kind is minikube with non docker driver and run minikube image load

OladapoAjala commented 2 years ago

@tejal29 thank you for the explanation, some follow up questions (affirmations):

  1. We'd be detecting & supporting drivers other than the docker driver
  2. If the current driver is docker just continue with the already existing flow
  3. Else load the appropriate image and run minikube image load
tejal29 commented 2 years ago

@tejal29 thank you for the explanation, some follow up questions (affirmations):

  1. We'd be detecting & supporting drivers other than the docker driver

yes that is correct.

  1. If the current driver is docker just continue with the already existing flow

yep. All things should work as is.

  1. Else load the appropriate image and run minikube image load

Exactly!