get ETCD version from kubernetes constants

medyagh commented 3 years ago

currently in https://github.com/medyagh/minikube/blob/a67a4ccbedd932f184b5713c70498dc434942621/pkg/minikube/bootstrapper/images/images.go#L86 we have

// etcd returns the image used for etcd
func etcd(v semver.Version, mirror string) string {
    // Should match `DefaultEtcdVersion` in:
    // https://github.com/kubernetes/kubernetes/blob/master/cmd/kubeadm/app/constants/constants.go
    ev := "3.4.13-0"

    switch v.Minor {
    case 17, 18:
        ev = "3.4.3-0"
    case 16:
        ev = "3.3.15-0"
    case 14, 15:
        ev = "3.3.10"
    case 12, 13:
        ev = "3.2.24"
    case 11:
        ev = "3.2.18"
    }

    // An awkward special case for v1.19.0 - do not imitate unless necessary
    if v.Equals(semver.MustParse("1.19.0")) {
        ev = "3.4.9-1"
    }

    return path.Join(kubernetesRepo(mirror), "etcd:"+ev)
}

instead we should use this map in kuberentes so we dont have to update this manually

https://github.com/kubernetes/kubernetes/blob/b58a7e233e06f5e4b58a637af8e36cb4f59c001b/cmd/kubeadm/app/constants/constants.go#L452


    // SupportedEtcdVersion lists officially supported etcd versions with corresponding Kubernetes releases
    SupportedEtcdVersion = map[uint8]string{
        13: "3.2.24",
        14: "3.3.10",
        15: "3.3.10",
        16: "3.3.17-0",
        17: "3.4.3-0",
        18: "3.4.3-0",
        19: "3.4.13-0",
        20: "3.4.13-0",
        21: "3.4.13-0",
        22: "3.4.13-0",
        23: "3.4.13-0",
    }

afbjorklund commented 3 years ago

It's not really in kubernetes but in kubeadm, and we can access these through config images list

$ ~/.minikube/cache/linux/v1.20.2/kubeadm config images list
k8s.gcr.io/kube-apiserver:v1.20.7
k8s.gcr.io/kube-controller-manager:v1.20.7
k8s.gcr.io/kube-scheduler:v1.20.7
k8s.gcr.io/kube-proxy:v1.20.7
k8s.gcr.io/pause:3.2
k8s.gcr.io/etcd:3.4.13-0

By default it will try to get latest: remote version is much newer: v1.21.1; falling back to: stable-1.20

Forgot what the reason was (but there was one) for doing it in code, and for duplicating all the constants.

afbjorklund commented 3 years ago

@medyagh : better fix all of them, if you fix one ?

// Pause returns the image name to pull for a given Kubernetes version
func Pause(v semver.Version, mirror string) string

// coreDNS returns the images used for CoreDNS
func coreDNS(v semver.Version, mirror string) string

// etcd returns the image used for etcd
func etcd(v semver.Version, mirror string) string

The original code is in cmd/kubeadm/app/images/images.go

https://github.com/kubernetes/kubernetes/blob/master/cmd/kubeadm/app/images/images.go (https://github.com/kubernetes/kubernetes/blob/master/cmd/kubeadm/app/constants/constants.go)

I think the problem was with accessing the earlier releases ?

    // DefaultEtcdVersion indicates the default etcd version that kubeadm uses
    DefaultEtcdVersion = "3.4.13-3"

    // CoreDNSVersion is the version of CoreDNS to be deployed if it is used
    CoreDNSVersion = "v1.8.0"

    // PauseVersion indicates the default pause image version for kubeadm
    PauseVersion = "3.4.1"

medyagh commented 3 years ago

Agree. Anyone picking up this task fix all verisons. To get vers through kubeadm

Srikrishnabh commented 3 years ago

@medyagh I would like to fix this.

vibecoder commented 3 years ago

/assign

Srikrishnabh commented 3 years ago

@vigothehacker If you dont mind can I work on this had already started, missed the assign command ?

Srikrishnabh commented 3 years ago

@medyagh : better fix all of them, if you fix one ?
// Pause returns the image name to pull for a given Kubernetes version
func Pause(v semver.Version, mirror string) string

// coreDNS returns the images used for CoreDNS
func coreDNS(v semver.Version, mirror string) string

// etcd returns the image used for etcd
func etcd(v semver.Version, mirror string) string
The original code is in cmd/kubeadm/app/images/images.go

https://github.com/kubernetes/kubernetes/blob/master/cmd/kubeadm/app/images/images.go (https://github.com/kubernetes/kubernetes/blob/master/cmd/kubeadm/app/constants/constants.go)

I think the problem was with accessing the earlier releases ?

yes to get the older release versions.

afbjorklund commented 3 years ago

I think the reason was that we wanted to know the versions before we downloaded kubeadm...

And that not all platforms were able to execute kubeadm, without wrapping it in docker or similar ?

One solution would be to make some kind of mapping table. That is, in markup instead of in code ?

---
v1.21.0:
    k8s.gcr.io/kube-apiserver: v1.21.0
    k8s.gcr.io/kube-controller-manager: v1.21.0
    k8s.gcr.io/kube-scheduler: v1.21.0
    k8s.gcr.io/kube-proxy: v1.21.0
    k8s.gcr.io/pause: 3.4.1
    k8s.gcr.io/etcd: 3.4.13-0
    k8s.gcr.io/coredns/coredns: v1.8.0
v1.20.0:
    k8s.gcr.io/kube-apiserver: v1.20.0
    k8s.gcr.io/kube-controller-manager: v1.20.0
    k8s.gcr.io/kube-scheduler: v1.20.0
    k8s.gcr.io/kube-proxy: v1.20.0
    k8s.gcr.io/pause: 3.2
    k8s.gcr.io/etcd: 3.4.13-0
    k8s.gcr.io/coredns: 1.7.0
v1.19.0:
    k8s.gcr.io/kube-apiserver: v1.19.0
    k8s.gcr.io/kube-controller-manager: v1.19.0
    k8s.gcr.io/kube-scheduler: v1.19.0
    k8s.gcr.io/kube-proxy: v1.19.0
    k8s.gcr.io/pause: 3.2
    k8s.gcr.io/etcd: 3.4.9-1
    k8s.gcr.io/coredns: 1.7.0

Note that you also have to supply --kubernetes-version, or it will auto-update (within each release).

vibecoder commented 3 years ago

/unassign

Srikrishnabh commented 3 years ago

/assign

Srikrishnabh commented 3 years ago

I think the reason was that we wanted to know the versions before we downloaded kubeadm...

And that not all platforms were able to execute kubeadm, without wrapping it in docker or similar ?

One solution would be to make some kind of mapping table. That is, in markup instead of in code ?
---
v1.21.0:
    k8s.gcr.io/kube-apiserver: v1.21.0
    k8s.gcr.io/kube-controller-manager: v1.21.0
    k8s.gcr.io/kube-scheduler: v1.21.0
    k8s.gcr.io/kube-proxy: v1.21.0
    k8s.gcr.io/pause: 3.4.1
    k8s.gcr.io/etcd: 3.4.13-0
    k8s.gcr.io/coredns/coredns: v1.8.0
v1.20.0:
    k8s.gcr.io/kube-apiserver: v1.20.0
    k8s.gcr.io/kube-controller-manager: v1.20.0
    k8s.gcr.io/kube-scheduler: v1.20.0
    k8s.gcr.io/kube-proxy: v1.20.0
    k8s.gcr.io/pause: 3.2
    k8s.gcr.io/etcd: 3.4.13-0
    k8s.gcr.io/coredns: 1.7.0
v1.19.0:
    k8s.gcr.io/kube-apiserver: v1.19.0
    k8s.gcr.io/kube-controller-manager: v1.19.0
    k8s.gcr.io/kube-scheduler: v1.19.0
    k8s.gcr.io/kube-proxy: v1.19.0
    k8s.gcr.io/pause: 3.2
    k8s.gcr.io/etcd: 3.4.9-1
    k8s.gcr.io/coredns: 1.7.0
Note that you also have to supply --kubernetes-version, or it will auto-update (within each release).

@afbjorklund I'm able to collect the image versions thanks for the inputs.

where do we want to stores this image to k8s version map, did you mean this will be some yaml and we use in code to load?

afbjorklund commented 3 years ago

I'm not sure if the best is to embed some kind of resource, or convert the lookup back into code again.

The original input is like:

$ curl -o kubeadm-linux-amd64-v1.13.0 https://storage.googleapis.com/kubernetes-release/release/v1.13.0/bin/linux/amd64/kubeadm
$ chmod +x kubeadm-linux-amd64-v1.13.0
$ ./kubeadm-linux-amd64-v1.13.0 --kubernetes-version=v1.13.0 config images list
k8s.gcr.io/kube-apiserver:v1.13.0
k8s.gcr.io/kube-controller-manager:v1.13.0
k8s.gcr.io/kube-scheduler:v1.13.0
k8s.gcr.io/kube-proxy:v1.13.0
k8s.gcr.io/pause:3.1
k8s.gcr.io/etcd:3.2.24
k8s.gcr.io/coredns:1.2.6

I converted it to some yaml markup like:

v1.13.0:
    k8s.gcr.io/kube-apiserver: v1.13.0
    k8s.gcr.io/kube-controller-manager: v1.13.0
    k8s.gcr.io/kube-scheduler: v1.13.0
    k8s.gcr.io/kube-proxy: v1.13.0
    k8s.gcr.io/pause: 3.1
    k8s.gcr.io/etcd: 3.2.24
    k8s.gcr.io/coredns: 1.2.6

It would also be possible to use go code:

    images := make(map[string]map[string]string)
    images["v1.13.0"] = map[string]string{
        "k8s.gcr.io/kube-apiserver":          "v1.13.0",
        "k8s.gcr.io/kube-controller-manager": "v1.13.0",
        "k8s.gcr.io/kube-scheduler":          "v1.13.0",
        "k8s.gcr.io/kube-proxy":              "v1.13.0",
        "k8s.gcr.io/pause":                   "3.1",
        "k8s.gcr.io/etcd":                    "3.2.24",
        "k8s.gcr.io/coredns":                 "1.2.6",
    }

Maybe @medyagh has some more thoughts, on what would be the best way to store these constants ?

afbjorklund commented 3 years ago

I made a PR to update the values for 1.21, while you think of the best solution for 1.22 and beyond...

Srikrishnabh commented 3 years ago

I'm not sure if the best is to embed some kind of resource, or convert the lookup back into code again.

The original input is like:

$ curl -o kubeadm-linux-amd64-v1.13.0 https://storage.googleapis.com/kubernetes-release/release/v1.13.0/bin/linux/amd64/kubeadm
$ chmod +x kubeadm-linux-amd64-v1.13.0
$ ./kubeadm-linux-amd64-v1.13.0 --kubernetes-version=v1.13.0 config images list
k8s.gcr.io/kube-apiserver:v1.13.0
k8s.gcr.io/kube-controller-manager:v1.13.0
k8s.gcr.io/kube-scheduler:v1.13.0
k8s.gcr.io/kube-proxy:v1.13.0
k8s.gcr.io/pause:3.1
k8s.gcr.io/etcd:3.2.24
k8s.gcr.io/coredns:1.2.6

I converted it to some yaml markup like:

v1.13.0:
    k8s.gcr.io/kube-apiserver: v1.13.0
    k8s.gcr.io/kube-controller-manager: v1.13.0
    k8s.gcr.io/kube-scheduler: v1.13.0
    k8s.gcr.io/kube-proxy: v1.13.0
    k8s.gcr.io/pause: 3.1
    k8s.gcr.io/etcd: 3.2.24
    k8s.gcr.io/coredns: 1.2.6

It would also be possible to use go code:

  images := make(map[string]map[string]string)
  images["v1.13.0"] = map[string]string{
      "k8s.gcr.io/kube-apiserver":          "v1.13.0",
      "k8s.gcr.io/kube-controller-manager": "v1.13.0",
      "k8s.gcr.io/kube-scheduler":          "v1.13.0",
      "k8s.gcr.io/kube-proxy":              "v1.13.0",
      "k8s.gcr.io/pause":                   "3.1",
      "k8s.gcr.io/etcd":                    "3.2.24",
      "k8s.gcr.io/coredns":                 "1.2.6",
  }

Maybe @medyagh has some more thoughts, on what would be the best way to store these constants ?

@medyagh any comments?

medyagh commented 3 years ago

@Srikrishnabh I agree with what @afbjorklund suggested this could be a map in our constants package we should have an automated tool to update that constants package file for example

make update-kubeadm-consts that will get the images list for that specific version of kubernetes and put it in our constants as a map

k8s-triage-robot commented 3 years ago

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue or PR as fresh with /remove-lifecycle stale
Mark this issue or PR as rotten with /lifecycle rotten
Close this issue or PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

Srikrishnabh commented 3 years ago

/remove-lifecycle stale

k8s-triage-robot commented 2 years ago

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue or PR as fresh with /remove-lifecycle stale
Mark this issue or PR as rotten with /lifecycle rotten
Close this issue or PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot commented 2 years ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue or PR as fresh with /remove-lifecycle rotten
Close this issue or PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

spowelljr commented 1 year ago

This currently creates the following output but it should not:

I0517 11:25:37.298578   24430 kubeadm.go:322] W0517 18:25:29.455933    1314 images.go:80] could not find officially supported version of etcd for Kubernetes v1.27.1, falling back to the nearest etcd version (3.5.7-0)

kubernetes / minikube

get ETCD version from kubernetes constants #11290