Open adietish opened 2 months ago
I am working on a patch as this configuration/setup looks rather common when a user has lots of clusters. It allows a user to split up the configuration into smaller chunks rather to have a single big kube config.
Patch seems rather simple (just looking, didnt code so far):
call io.fabric8.kubernetes.client.Config#mergeKubeConfigContents
for each file that's present in the system variable, making sure that overrides/merges happen like in kubectl
For my own documentation:
if the KUBECONFIG
env variable is set (~/.kube/config:<additional file>:<additional file>
), kubectl
is using the first file in the list and adds
that are found in the additional files. The code for it is in client_go
if the
KUBECONFIG
env variable is set (~/.kube/config:<additional file>:<additional file>
),kubectl
is using the first file in the list and adds
@adietish : I'm sorry for replying late, after reading your comment I realized we had merged something similar a few years ago. However, I'm not sure whether it's completely aligned with kubectl
behavior or not.
@rohanKanojia thx for the pointer, I had already looked into this. This PR only takes the 1st of the given files into account. All further ones are ignored. See here: https://github.com/fabric8io/kubernetes-client/pull/1306/files#diff-ee58b9555742a6e84ec59a74115410be46af05081afae5cf79a4516283ce37c7R444 There's a 2nd issue which add details to the logging warning but didnt' change the handling: #2174 (see https://github.com/fabric8io/kubernetes-client/pull/2181/files#diff-ee58b9555742a6e84ec59a74115410be46af05081afae5cf79a4516283ce37c7R477)
LOGGER.warn("Found multiple Kubernetes config files [{}], using the first one: [{}].
If not desired file, please change it by doing `export KUBECONFIG=/path/to/kubeconfig` on Unix systems
or `$Env:KUBECONFIG=/path/to/kubeconfig` on Windows.", fileNames, fileNames[0]);
What I want to improve is to allow several files to be used. In https://github.com/quarkusio/quarkus/issues/8946#issuecomment-621144866 @manusa brought up how such a setup would work: Reading the kubectl docs, testing and reading of kubectl code suggests the following behaviour:
current-context
. None of the additional files in KUBECONFIG
env var can set the current-context
. (tested)KUBECONFIG
contribute contexts
, clusters
and users
. https://github.com/redhat-developer/intellij-kubernetes/issues/779 provides an exemplary setup where the 1st file sets the current-context
but has no contexts, nor clusters, nor users. Then there are separate files for each cluster (ex. minikube.yml, sandbox.yml, etc.) which contribute a context, a cluster and a user.
Reading the kubectl docs, testing and reading of kubectl code suggests the following behaviour:
* only the first file sets the `current-context`. None of the additional files in `KUBECONFIG` env var can set the `current-context`. (tested) * all the additional files listed in `KUBECONFIG` contribute `contexts`, `clusters` and `users` .
This seems fine.
Implementation should be quite easy for merging. However, I think there are several places where the underlying configuration is updated (token refresh, etc.). These might be the tricky parts, since we'll probably need to traverse all configs until the config containing the token is found and updated.
@manusa thx for the pointer, I found https://github.com/fabric8io/kubernetes-client/blob/342fb8ac941ff10c8bedcb5824f9f2d33a86d341/kubernetes-client-api/src/main/java/io/fabric8/kubernetes/client/utils/OpenIDConnectionUtils.java#L188-L225 Is this what you were referring to?
I have to specify my suggestion in https://github.com/fabric8io/kubernetes-client/issues/6240#issuecomment-2286276111 according to the Kubectl docs:
kubeconfigs listed in the KUBECONFIG
env var are fully merged while values in a latter file will not override a prior one. First in wins.
Never change the value or map key. Example: Preserve the context of the first file to set current-context. Example: If two files specify a red-user, use only values from the first file's red-user. Even if the second file has non-conflicting entries under red-user, discard them.
And then, when using kubectl --kubeconf <file>
only a single file is allowed (mo merging happens).
If the --kubeconfig flag is set, use only the specified file. Do not merge. Only one instance of this flag is allowed.
Is your enhancement related to a problem? Please describe
One can provide kubectl with multiple kube config files via the
KUBECONFIG
env variable. Kubectl would merge those and let the user use them. ex.export KUBECONFIG=${HOME}/.kube/config:${HOME}/.kube/minikube.yaml:${HOME}/.kube/sandbox.yaml
~/.kube/config
with onlycurrent-context
pointing to a context that's defined in further file~/.kube/minikube.yml
~/.kube/sandbox.yml
In kubectl I can then have
config
pointing to minikube or sandbox, both would work. In kubernetes-client, only the 1st file is used which then fails because it is missing url, user, etc.Describe the solution you'd like
It would be great if kubernetes-client could merge these files into one config like kubectl does.
Describe alternatives you've considered
No response
Additional context
No response