You should be able to tell Tilt about your cluster if Tilt can't infer things

[jazzdan]

Right now Tilt uses some pretty simple heuristics to decide what kind of Kubernetes cluster you have:

1. Strict matching on the name of the cluster, like with minikube.
2. Fuzzy matching on the name of the cluster, like with GKE
3. Path to kubeconfig, like with k3d.
4. Fuzzy matching of path to kubeconfig, like with KIND.

Tilt uses this information to decide whether it can use an internal image registry, among other things. It's important for the Tilt user experience that we use an internal image registry whenever possible since that speeds up image push times.

Unfortunately it's easy for a user to inadvertently break Tilt's cluster inference code by renaming clusters or moving kube configs.

We should provide some sort of escape hatch to tell Tilt to consider cluster named X to be minikube/microk8s or something. I'm not entirely sure yet. I'm opening this issue to spur discussion.

[adamf]

My use case for this that I have multiple developers using minikube+tilt to develop a product. The product has some monitoring/logging code in that tags the metrics with the cluster name. If all the clusters are named minikube regardless of developer, it's hard to tell one developer cluster from another, slowing down debugging and crowding up the dashboards.

I can rename my minikube clusters via minikube -p <newclustername> or via configs, and I can then set my context name in the Tiltfile to match, but due to the heuristics mentioned in jazzdan's comment above Tilt thinks I am working on a remote cluster because the environment test in Tilt doesn't recognize my renamed cluster as being local and on minikube.

[nicks]

Here's what my kubeconfig looks like if I start minikube with minikube -p nick

apiVersion: v1
clusters:
- cluster:
    certificate-authority: /home/nick/.minikube/ca.crt
    server: https://192.168.39.37:8443
  name: nick
- context:
    cluster: nick
    user: nick
  name: nick
current-context: nick
kind: Config
preferences: {}
users:
- name: nick
  user:
    client-certificate: /home/nick/.minikube/client.crt
    client-key: /home/nick/.minikube/client.key

so maybe a simple way to fix this is just to check if your kubeconfig is using a certificate authority under $HOME/.minikube

I'm not against adding a way to specify the cluster env. But I do think it's a strong requirement that Tilt should auto-detect cluster types when it can. If you're a new Tilt user, it wouldn't occur to you to go searching for a minikube env flag.

[adamf]

+1 for the autodetection; might be good to also include a k8s_environment() in the tiltfile.

[adamf]

Looking at the detection logic, it might be simplest to say "we detect environment based on the prefix of the cluster name" - Tilt does this for most but not all of the cluster types. You'd change:

if Env(cn) == EnvMinikube {
    return EnvMinikube

to:

if cn == strings.HasPrefix(cn, string(EnvMinikube)) {
   return EnvMinikube

And change the few others that don't do the prefix matching to be prefix matched. At least it would be consistent.

[DblK]

I suggest to add a command for tilt to let it know that a cluster should be considered as a local one. For example using k3os in qemu or elsewhere, I wish to have it as local. (Taken from #3062)

Here are my suggestions for this:

• tilt set localenv my-context ==> Set a specific k8s context to be a local one (ignoring push)
• tilt get localenv / tilt ls localenv => Get/List localenv or in another way:
• tilt local set my-context
• tilt local ps / tilt local get / tilt local ls

Pick the ones you find the best ;)