cruise-automation / isopod

An expressive DSL and framework for Kubernetes configuration without YAML
Apache License 2.0
482 stars 32 forks source link

Support kubeconfig chaining etc. for onprem #18

Open wosc opened 4 years ago

wosc commented 4 years ago

It would be nice if isopod supported a) assuming a default of $HOME/.kube/config (I have to explicitly pass that as -kubeconfig=, else I get Failed to build kube rest config for k8s vendor <onprem: {}>) b) reading the $KUBECONFIG environment variable c) chaining several kube config files (e.g. define clusters/contexts in $HOME/.kube/config, then select the current-context in a project-local config file)

cxuu commented 4 years ago

I think your point a) and b) are straightforward and reasonable. I will make an internal ticket.

For point c), we actually made the conscious decision to move away from the current cluster context in kubeconfig file. Isopod targets multiple clusters and allows the users the select a subset of them with a custom implementation of the clusters() function in main.ipd. Using current-context can be error-prone, because from our prior experience, users could easily forget to set it before rollout or unset it after rollout. Set/unset is necessary since we are targeting multiple clusters.

To summarize, Isopod uses kubeconfig file only for authentication to on-prem clusters, not for cluster selection or other managed k8s offerings (because we do not want to manage token renewal).