Open tamalsaha opened 4 years ago
I tend to use https://github.com/GoogleCloudPlatform/kubectl-dispatcher installed via gcloud components install kubectl
as my system kuebctl, it automatically dispatches over multiple versions so you don't have to keep reinstalling.
I'm not sure how we should best match this into kind though, especially since kind is meant to support completely arbitrary k8s commits.
@seans3 is it feasible to drop binaries for specific commits in some directory and have the dispatcher match as closely as possible?
the node images do have kubectl, on linux you can just copy it out to the host with a one-liner, but mac / windows are problematic.
The kubectl-dispatcher
currently only matches the version major/minor, falling back to the dispatcher's default version.
Some background: the kubectl dispatcher
is kubectl
with a simple query/dispatch happening before every kubectl
command. The kubectl dispatcher
asks the APIServer what version it is, then it executes a version of kubectl
(in the same directory) that has the same major/minor version. This version is named kubectl.<MAJOR>.<MINOR>
. Example: kubectl.1.15
. If the kubectl dispatcher
does not find this versioned kubectl
, it just drops through to execute at it's default version. This is what the kubectl/gcloud distribution looks like:
$ ls -l /usr/local/google-cloud-sdk/bin/kubectl*
-rwxr-xr-x 1 sean sean 43086784 Feb 5 19:29 /usr/local/google-cloud-sdk/bin/kubectl
-rwxr-xr-x 1 sean sean 39271904 Oct 15 2019 /usr/local/google-cloud-sdk/bin/kubectl.1.13
-rwxr-xr-x 1 sean sean 43119424 Dec 11 05:11 /usr/local/google-cloud-sdk/bin/kubectl.1.14
-rwxr-xr-x 1 sean sean 42993696 Jan 18 16:08 /usr/local/google-cloud-sdk/bin/kubectl.1.15
-rwxr-xr-x 1 sean sean 42889216 Jan 18 16:04 /usr/local/google-cloud-sdk/bin/kubectl.1.16
-rwxr-xr-x 1 sean sean 43491328 Jan 18 16:03 /usr/local/google-cloud-sdk/bin/kubectl.1.17
@BenTheElder to answer your question about dropping binaries into a location: it would have to look like this previous organization. It requires a special kubectl
with the dispatch mechanism, and it requires versioned kubectl
binaries named according to the convention in the same directory as the kubectl dispatcher
.
@BenTheElder Please let me know how I can help. I can probably put a kubectl dispatcher
tarball in a google cloud bucket for download.
thanks, I think we might need to consider also placing it somewhere like github, kind is generally workable in china and has an active userbase there but we've found with our nightly builds that GCS cannot be reached behind GFW.
how would the dispatcher to tiebreak kubectl1.15.5
and kubectl1.15.5-something
binaries if we drop more in there?
also not sure how we avoid infinite versions.
maybe we focus on installing release versions and just assume that the api is not supposed to break in 1.X.
I'm not sure when we'll get to this with some other big changes still settling but I really like this idea, this has been a fairly major pain point for me as well.
I'm going to sync with seans3 soon to discuss what it might look like to support installing the dispatcher (and installing more kubectl binaries) because I think that's the least friction way to go, you wouldn't need to switch between versions manually, just make sure they're installed and then it just matches the server :-)
Similar tools for Helm https://github.com/tokiwong/helm-switcher
We have this in minikube kubectl
, which was inspired by microk8s.kubectl
.
https://github.com/kubernetes/minikube/blob/master/cmd/minikube/cmd/kubectl.go
Basically we download the correct version to the cache and run it from there.
Here's another nice approach: https://apps.0install.net/kubernetes/kubectl.xml
kind has the kubectl
binary inside the node image, and it matches the other kubernetes componentes in the node
docker exec -it kind-control-plane /usr/bin/kubectl version --kubeconfig /etc/kubernetes/admin.conf
Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.2", GitCommit:"52c56ce7a8272c798dbc29846288d7cd9fbae032", GitTreeState:"clean", BuildDate:"2020-04-30T20:18:51Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.2", GitCommit:"52c56ce7a8272c798dbc29846288d7cd9fbae032", GitTreeState:"clean", BuildDate:"2020-04-30T20:19:45Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
a quick hack
aojea@host:~/go/src/sigs.k8s.io/kind$ export KUBECTL="docker exec -it kind-control-plane /usr/bin/kubectl --kubeconfig /etc/kubernetes/admin.conf"
aojea@host:~/go/src/sigs.k8s.io/kind$ $KUBECTL get services
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP fd00:10:96::1 <none> 443/TCP 90m
Usually, I will directly copy kubectl to the local after using KIND to start the cluster
(MoeLove) ➜ ~ docker cp kind-control-plane:/kind/bin/kubectl ~/bin/kubectl
(MoeLove) ➜ ~ ~/bin/kubectl version
Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.2", GitCommit:"52c56ce7a8272c798dbc29846288d7cd9fbae032", GitTreeState:"clean", BuildDate:"2020-04-30T20:18:51Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.2", GitCommit:"52c56ce7a8272c798dbc29846288d7cd9fbae032", GitTreeState:"clean", BuildDate:"2020-04-30T20:19:45Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
That only works on Linux @aojea @tao12345666333.
We can pretty easily do something like minikube kubectl, however I think the dispatcher is preferable so as to correctly handle multiple contexts & versions without reinventing the wheel.
The biggest trick to consider is that we support unreleased kubernetes versions / arbitrary commits. We'll also need to improve the dispatcher hosting, and determine where to install this on the host.
That only works on Linux @aojea @tao12345666333.
We can pretty easily do something like minikube kubectl, however I think the dispatcher is preferable so as to correctly handle multiple contexts & versions without reinventing the wheel.
The biggest trick to consider is that we support unreleased kubernetes versions / arbitrary commits. We'll also need to improve the dispatcher hosting, and determine where to install this on the host.
Can we wrap with kind the inner kubectl?
kind kubectl --name kind-cluster ...
Not cleanly due to -f ...
On Sat, Jun 6, 2020, 06:22 Antonio Ojea notifications@github.com wrote:
That only works on Linux @aojea https://github.com/aojea @tao12345666333 https://github.com/tao12345666333.
We can pretty easily do something like minikube kubectl, however I think the dispatcher is preferable so as to correctly handle multiple contexts & versions without reinventing the wheel.
The biggest trick to consider is that we support unreleased kubernetes versions / arbitrary commits. We'll also need to improve the dispatcher hosting, and determine where to install this on the host.
Can we wrap with kind the inner kubectl? kind kubectl --name kind-cluster ...
— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/kubernetes-sigs/kind/issues/1502#issuecomment-640060460, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAHADKZWJYID6XFSV2A7ZGTRVI7H7ANCNFSM4MONV5SQ .
/remove-kind design
/kind feature
kind/design
is migrated to kind/feature
, see https://github.com/kubernetes/community/issues/6144 for more details
Hi, this label was only supposed to be migrated for kubernetes/kubernetes. Please see: https://groups.google.com/g/kubernetes-dev/c/f6W68eS8xo0
Eh, whatever 🤷♂️
What would you like to be added: It will be useful if kind could also kubectl, kustomize etc.
Why is this needed: kubectl, etc. needs to match the version of k8s. Right now, it is a clumsy manual process and kind of pain to add in a CI env.
Something like https://github.com/nutellinoit/kubenvz could be built into kind, it will be even more awesome.