search

kubernetes-sigs / kind

Kubernetes IN Docker - local clusters for testing Kubernetes
https://kind.sigs.k8s.io/
Apache License 2.0
13.43k stars 1.55k forks source link

Output the default node version #2612

Closed afbjorklund closed 1 year ago

afbjorklund commented 2 years ago

What would you like to be added:

It could be informative, to be able to output the kubernetes version of the default node. For instance for downloading kubectl.

Why is this needed:

I was making installers for kind and k3d (for some reason), and noticed there was a difference in the version output:

$ kind version
kind v0.11.1 go1.16.4 linux/amd64
$ k3d version
k3d version v5.2.2
k3s version v1.21.7-k3s1 (default)

So it could be nice if the kind version had outputted some similar information, about the default node image:

kubernetes v1.21.1 (default)

Currently it seems that it is only in defaults.Image (pkg/apis/config/defaults/image.go) but not in the CLI ?

The information is printed on the GitHub release page, but not inside the tool itself. But it could be useful...

Here is a workaround: (or hack)

strings $(which kind) | grep -o 'kindest/node:v[0-9.]*' | sed -e 's,kindest/node:,,'

I know you can override the node image (and thus k8s version). This was more for the case when you didn't.

BenTheElder commented 2 years ago

I'm sure folks are capturing the output of this version command by now and will be annoyed if it becomes multi-line, i'm not sure how best to expose this in the CLI. Perhaps a new command or kind version with increased verbosity? 🤔 Or we could just add it, but ... 🤔

For use cases other than installers, I recommend either not relying on the default version (use one of the published images explicitly) or going all in on defaulting and not setting the field (which we will default everywhere, even e.g. with multiple nodes), and detecting the version at runtime in k8s tooling (poke api-server/call kubectl).

afbjorklund commented 2 years ago

This was the previous default: kube_version=$(curl -L -s https://dl.k8s.io/release/stable.txt)

But it was starting to give warnings, so I was thinking that maybe I should sync them instead ?

The alternative would be to use some tool that syncs them automatically (like in minikube kubectl).

Such as this one: https://github.com/flavio/kuberlr

Seems like there are no warnings, after the upgrade:

Client Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.3", GitCommit:"816c97ab8cff8a1c72eccca1026f7820e93e0d25", GitTreeState:"clean", BuildDate:"2022-01-25T21:25:17Z", GoVersion:"go1.17.6", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.1", GitCommit:"86ec240af8cbd1b60bcc4c03c20da9b98005b92e", GitTreeState:"clean", BuildDate:"2022-01-06T22:13:50Z", GoVersion:"go1.17.5", Compiler:"gc", Platform:"linux/amd64"}

Looks like "latest" does not mean the same as github:

kind v0.12.0-alpha+666b3368ea5659 go1.17.6 linux/amd64

https://github.com/kubernetes-sigs/kind/releases/latest

BenTheElder commented 2 years ago

But it was starting to give warnings, so I was thinking that maybe I should sync them instead ?

To sync kubectl versions, I'd really like to see https://github.com/GoogleCloudPlatform/kubectl-dispatcher upstreamed, gcloud ships with this and it solves the version really cleanly (so you also don't have to keep reinstalling when using different clusters).

There's some discussion of this in another thread ... I think nobody has had the time but there's no major opposition last I checked.

BenTheElder commented 2 years ago

https://github.com/kubernetes-sigs/kind/issues/1502 found it.

Looks like "latest" does not mean the same as github:

ah, the nightly binary downloads? those are still un-advertised for a reason :+) originally they were an internal detail of kubernetes's CI. we could rename them with some :effort:

afbjorklund commented 2 years ago

Forgot to link to the actual yaml, if anyone is interested...

I can get the latest version from the github API, no worries.

I don't think it will go upstream anyway, prefer using kubeadm and k3s.

The main reason was that people wanted Kubernetes, without Docker.

BenTheElder commented 2 years ago

Thanks so much for sharing these!

I'm open to suggestions for how we might add this or maybe:

Perhaps a new command or kind version with increased verbosity? 🤔 Or we could just add it, but ... 🤔

but I don't think I'm going to have much time for this myself soon, we still need to find time to finish the fuse issue and get a release. I'm personally investing most of my efforts in fixing sig k8s infra exceeding our GCP credit budget at the moment :this-is-fine: and looking for more folks to help run SIG Testing ...

cc @tamalsaha ^^^ lima samples I mentioned the other day https://twitter.com/BenTheElder/status/1489060668003086336 (sorry I couldn't find your twitter if you had one @afbjorklund, so I just mentioned the slack thread and linked to your github)

afbjorklund commented 2 years ago

It was supposed to work out-of-the-box when using "docker.yaml" to start rootless docker. Assuming that the user installs docker and kubectl themselves (brew install)

limactl start https://raw.githubusercontent.com/lima-vm/lima/master/examples/docker.yaml

But there were some issues with that, so made an all-in-one (with normal docker) with all the client tools also on the VM (accessible with SSH). So you can use lima kubectl with it.

limactl start https://raw.githubusercontent.com/afbjorklund/lima/kind/examples/kind.yaml
afbjorklund commented 1 year ago

I added some more nodes to the examples, to make it more "interesting".

kind.yaml

NAME                 STATUS   ROLES           AGE   VERSION
kind-control-plane   Ready    control-plane   32s   v1.26.0
kind-worker          Ready    <none>          12s   v1.26.0
kind-worker2         Ready    <none>          11s   v1.26.0
kind-worker3         Ready    <none>          12s   v1.26.0

k3d.yaml

NAME                     STATUS   ROLES                  AGE   VERSION
k3d-mycluster-agent-0    Ready    <none>                 23s   v1.24.4+k3s1
k3d-mycluster-agent-2    Ready    <none>                 22s   v1.24.4+k3s1
k3d-mycluster-agent-1    Ready    <none>                 22s   v1.24.4+k3s1
k3d-mycluster-server-0   Ready    control-plane,master   28s   v1.24.4+k3s1

But will use the code from above, to get the version to use for kubectl.

$ limactl shell kind kubectl version
WARNING: This version information is deprecated and will be replaced with the output from kubectl version --short.  Use --output=yaml|json to get the full version.
Client Version: version.Info{Major:"1", Minor:"26", GitVersion:"v1.26.0", GitCommit:"b46a3f887ca979b1a5d14fd39cb1af43e7e5d12d", GitTreeState:"clean", BuildDate:"2022-12-08T19:58:30Z", GoVersion:"go1.19.4", Compiler:"gc", Platform:"linux/amd64"}
Kustomize Version: v4.5.7
Server Version: version.Info{Major:"1", Minor:"26", GitVersion:"v1.26.0", GitCommit:"b46a3f887ca979b1a5d14fd39cb1af43e7e5d12d", GitTreeState:"clean", BuildDate:"2022-12-20T03:35:13Z", GoVersion:"go1.19.4", Compiler:"gc", Platform:"linux/amd64"}
$ limactl shell k3d kubectl version
WARNING: This version information is deprecated and will be replaced with the output from kubectl version --short.  Use --output=yaml|json to get the full version.
Client Version: version.Info{Major:"1", Minor:"24", GitVersion:"v1.24.4", GitCommit:"95ee5ab382d64cfe6c28967f36b53970b8374491", GitTreeState:"clean", BuildDate:"2022-08-17T18:54:23Z", GoVersion:"go1.18.5", Compiler:"gc", Platform:"linux/amd64"}
Kustomize Version: v4.5.4
Server Version: version.Info{Major:"1", Minor:"24", GitVersion:"v1.24.4+k3s1", GitCommit:"c3f830e9b9ed8a4d9d0e2aa663b4591b923a296e", GitTreeState:"clean", BuildDate:"2022-08-25T03:45:26Z", GoVersion:"go1.18.1", Compiler:"gc", Platform:"linux/amd64"}