support minikube with --vm-driver=none

[nicks]

Currently, if you start minikube with --vm-driver=none, Tilt does not work properly.

Seems to be some problem with how it's using minikube docker-env, needs investigation

Reported by Jonas in the k8s.slack.io #tilt channel

[JohnnyCrazy]

Just to shed some more light here:

When running tilt doctor, this is the output with a vm-driver=none cluster:

➜ tilt doctor
Tilt: v0.8.9, built 2019-06-07
System: linux-amd64
---
Docker
- Host: Error: ProvideDockerEnv: Could not read docker env from minikube.
Did you forget to run `minikube start`?
X The docker host is currently not running

- Version: Error: ProvideDockerEnv: Could not read docker env from minikube.
Did you forget to run `minikube start`?
X The docker host is currently not running

- Builder: Error: ProvideDockerEnv: Could not read docker env from minikube.
Did you forget to run `minikube start`?
X The docker host is currently not running

---
Kubernetes
- Env: minikube
- Context: minikube
- Cluster Name: minikube
- Namespace: default
- Container Runtime: docker
- Version: v1.14.2
---
Thanks for seeing the Tilt Doctor!
Please send this info along when filing bug reports. 💗

Two problems have to be solved:

• With vmdriver=none, minikube has to be started with elevated rights sudo. Thus, running just minikube docker-env will not find the running cluster. I'm not entirely sure how we could solve this one...
• sudo minikube docker-env reports 'none' driver does not support 'minikube docker-env' command. In this case, tilt should fall back to the hosts docker installation IMO?

[nicks]

a simple thing to do might be to check if the machine id in kubectl get nodes -o json matches the current machine id. And if that's true, we skip the minikube docker-env call because we know it has to be the current env.

[nicks]

If someone wants to contribute a PR for this, we'd accept it.

I'm concerned that --vm-driver=none is designed for the case "I built my own VM", NOT "I want to do local k8s development without a VM". And that the minikube team may make changes that break the second case. See:

https://github.com/kubernetes/minikube/blob/master/docs/vmdriver-none.md#can-the-none-driver-be-used-outside-of-a-vm

I wonder if we should be endorsing microk8s more strongly for the second case in the Tilt documentation

[dev-inigmas]

In the meantime.... Is there any way around this issue? I'm having difficulty on my development hardware getting Minikube with either KVM or VirtualBox to perform well enough to get anything done. And since Microk8s dropped Docker, Argo doesn't seem to work well with it anymore. So at this point, it really seems like developing using Minikube without a VM is the best setup for me.

Thanks.

[JohnnyCrazy]

Yea I got around it using a custom_build script, basically this:

custom_build('api',
  'docker build -t $EXPECTED_REF backend -f backend/dev.Dockerfile',
  ['backend'],
  ignore=[
    './backend/deps', './backend/_build', './backend/.git'
  ],
  disable_push=True,
  live_update=[
    fall_back_on(['./backend/mix.exs', './backend/mix.lock', './backend/config/']),
    sync('./backend', '/code')
  ]
)

Small hint regarding dev setup, I'm currently running a very good setup with k3d/k3s. Runs in docker, no vm required, persistency supported and low memory/cpu footprint. Have a look if you're not statisfied with minikube.

EDIT:

And rename your minikube cluster, so it's a normal one and not detected as minikube

[dev-inigmas]

How do I rename it? Sorry, still relatively new to the Kubernetes world.

[JohnnyCrazy]

How do I rename it? Sorry, still relatively new to the Kubernetes world.

Open your ~/.kube/config file and rename the respective keys of context, cluster and current-context.