Tasks to bump to Kubernetes v1.29

[killianmuldoon]

This issue is tracking the tasks that should be implemented after the Kubernetes minor release has been released.

Tasks

Prerequisites:

• [x] Decide which Cluster API release series will support the new Kubernetes version
  • If feasible we usually cherry-pick the changes back to the latest release series.

Supporting managing and running on the new Kubernetes version

This section contains tasks to update our book, e2e testing and CI to use and test the new Kubernetes version as well as changes to Cluster API that we might have to make to support the new Kubernetes version. All of these changes should be cherry-picked to all release series that will support the new Kubernetes version.

• [x] Modify quickstart and CAPD to use the new Kubernetes release:
  • Bump the Kubernetes version in:
  • test/*: search for occurrences of the previous Kubernetes version
  • Tiltfile
  • Ensure the latest available kind version is used (including the latest images for this kind release)
  • Add new images in the kind mapper.go.
    • See the kind releases page for the list of released images.
  • Set new default image for the test framework
  • Verify the quickstart manually
  • Bump InitWithKubernetesVersion and WorkloadKubernetesVersion in clusterctl_upgrade_test.go
  • Note: Only bump for Cluster API versions that will support the new Kubernetes release.
  • Prior art: #9160
• [x] Ensure the jobs are adjusted to provide test coverage according to our support policy:
  • For the main branch:
  • periodics:
    • Drop the oldest upgrade job as the oldest Kubernetes minor version is now out of support.
    • Add new upgrade job which upgrades from the previous to the new Kubernetes version.
  • periodics & presubmits:
    • Bump KUBERNETES_VERSION_MANAGEMENT of the e2e-mink8s job to the new minimum supported management cluster version.
    • Bump KUBEBUILDER_ENVTEST_KUBERNETES_VERSION of the test-mink8s jobs to the new minimum supported management cluster version.
    • Adjust the -latest upgrade job to upgrade from the new Kubernetes to the next Kubernetes version.
  • For the release branch of the latest supported Cluster API minor release:
  • periodics & presubmits:
    • Adust the -latest upgrade jobs to upgrade to the new Kubernetes version instead of latest.
  • Note: Also check if ETCD_VERSION_UPGRADE_TO or COREDNS_VERSION_UPGRADE_TO needs to change for the upgrades jobs to the new or next Kubernetes version.
  • For etcd, see the DefaultEtcdVersion kubeadm constant: e.g. for v1.28.0
  • For coredns, see the CoreDNSVersion kubeadm constant:e.g. for v1.28.0
  • Prior art: https://github.com/kubernetes/test-infra/pull/30347 https://github.com/kubernetes/test-infra/pull/30406 https://github.com/kubernetes/test-infra/pull/30407
• [x] Update book:
  • Update supported versions in versions.md
  • Update job documentation in jobs.md
  • Prior art: #9161
• [x] Issues specific to the Kubernetes minor release:
  • [x] RBAC kubeadm:cluster-admins must be applied before CAPI triggers the upgrade to kubeadm 1.29 machines. (assuming change is merged for 1.29. https://github.com/kubernetes-sigs/cluster-api/pull/9682

Using new Kubernetes dependencies

This section contains tasks to update Cluster API to use the latest Kubernetes Go dependencies and related topics like using the right Go version and build images. These changes are only made on the main branch. We don't need them in older releases as they are not necessary to manage workload clusters of the new Kubernetes version or run the Cluster API controllers on the new Kubernetes version.

• [x] Ensure there is a new controller-runtime minor release which uses the new Kubernetes Go dependencies.

• [x] Update our Prow jobs for the main branch to use the correct kubekins-e2e image

  • It is recommended to have one PR for presubmit and one for periodic jobs to reduce the risk of breaking the periodic jobs.
  • PRs: https://github.com/kubernetes/test-infra/pull/31450 https://github.com/kubernetes/test-infra/pull/31450
  • Prior art: presubmit jobs: https://github.com/kubernetes/test-infra/pull/27311
  • Prior art: periodic jobs: https://github.com/kubernetes/test-infra/pull/27326

• [x] Bump the Go version in Cluster API: (if Kubernetes is using a new Go minor version)

  • Search for the currently used Go version across the repository and update it
  • We have to at least modify it in: hack/ensure-go.sh, .golangci.yml, cloudbuild*.yaml, go.mod, Makefile, netlify.toml, Tiltfile
  • PR: #9900
  • Prior art: #7135

• [x] Bump controller-runtime #9964

• [x] Bump controller-tools #9987

• [x] Bump the Kubernetes version used in integration tests via KUBEBUILDER_ENVTEST_KUBERNETES_VERSION in Makefile

  • Note: This PR should be cherry-picked as well. It is part of this section as it depends on kubebuilder/controller-runtime releases and is not strictly necessary for Supporting managing and running on the new Kubernetes version.
  • PRs: #10013 #10014
  • Prior art: #7193

• [x] Bump conversion-gen via CONVERSION_GEN_VER in Makefile

  • PR: #10012
  • Prior art: #7118

• [x] ensure versions.md on main & release-1.6 is up-to-date after https://github.com/kubernetes-sigs/cluster-api/pull/9971

  • PR: https://github.com/kubernetes-sigs/cluster-api/pull/10024

[killianmuldoon]

Created now mainly to keep track of https://github.com/kubernetes/enhancements/pull/4302

/triage accepted

[chrischdi]

/assign

[furkatgofurov7]

@chrischdi Hey, one question: am I correct to assume that once this is completed, we will be supporting k8s 1.29 in the release-1.6 branch, i.e starting from v1.6.x, although there is a note in the book:

When a new Kubernetes minor release is available, we will try to support it in an upcoming Cluster API patch release (although only in the latest supported Cluster API minor release). But this depends on the changes made in Kubernetes, if the corresponding required changes in Cluster API are too invasive we won’t backport the support and users have to wait for the next Cluster API minor release.

meaning it might not be the case always?

[chrischdi]

Yes it is. With this issue we should also add the necessary jobs for the v1.6 release to get test coverage.

Especially via this part:

• For the release branch of the latest supported Cluster API minor release:
  • periodics & presubmits:
    • Adust the -latest upgrade jobs to upgrade to the new Kubernetes version instead of latest.

If any fixes are required they should get cherry-picked and released with the next round of patch releases (although normally there is no fix necessary).

[chrischdi]

Also note:

All of these changes should be cherry-picked to all release series that will support the new Kubernetes version.

[kashifest]

@chrischdi is there any timeline for the support for k8s v1.29 in CAPI, I think k8s v1.29 lands tomorrow, so can we expect the support in CAPI branches before the xmas or after? And also if theres any other blocker for example controller-runtime release? Meanwhile if any support is needed for the subtasks listed here in the issue, we can help.

[chrischdi]

@chrischdi is there any timeline for the support for k8s v1.29 in CAPI, I think k8s v1.29 lands tomorrow, so can we expect the support in CAPI branches before the xmas or after? And also if theres any other blocker for example controller-runtime release? Meanwhile if any support is needed for the subtasks listed here in the issue, we can help.

I want this to land (if possible) before xmas, however a supporting release will be v1.6.1 which is planned for January afaik.

[chrischdi]

Prerequisites:

• [ ] Decide which Cluster API release series will support the new Kubernetes version
  • If feasible we usually cherry-pick the changes back to the latest release series.

All work will be cherry-picked to release-1.6 so hopefully the upcoming release v1.6.1 will be the first one officially supporting Kubernetes v1.29

[chrischdi]

Sumarizing the above open tasks:

Supporting managing and running on the new Kubernetes version

• [x] (@chrischdi) Modify quickstart and CAPD to use the new Kubernetes release:

  • ~Waiting for kubernetes release and kind images~
  • ~Waiting for~ https://github.com/kubernetes-sigs/cluster-api/pull/9799
  • PR: https://github.com/kubernetes-sigs/cluster-api/pull/9872
  • PR release-1.6: https://github.com/kubernetes-sigs/cluster-api/pull/9890

• [x] (@chrischdi) Ensure the jobs are adjusted to provide test coverage according to our support policy:

  • Waiting for (above) https://github.com/kubernetes-sigs/cluster-api/pull/9872
  • Draft PR: https://github.com/kubernetes/test-infra/pull/31441

• [x] (@chrischdi) Update book:

  • ~Waiting for~ https://github.com/kubernetes-sigs/cluster-api/pull/9867
  • Waiting for https://github.com/kubernetes/test-infra/pull/31441

  • 9897

• [x] Issues specific to the Kubernetes minor release:

  • no open todo's currently, but we will see when the tests are running

Using new Kubernetes dependencies

After release of v1.6.1 (and if v1.6.1 supports Kubernetes v1.29):

• [x] bump WorkloadKubernetesVersion for test When testing clusterctl upgrades using ClusterClass (v1.6=>current) [ClusterClass]

After release-28 kubekins image includes go 1.21:

• [-] bump golang on release-1.6 and release-1.5 branches
  • Moved to: https://github.com/kubernetes-sigs/cluster-api/issues/10025

[kashifest]

@chrischdi are waiting for controller-runtime and controller-tools release on this ?

[sbueringer]

Yes.

[kashifest]

Do we have any expected timeline ?

[sbueringer]

No

[sbueringer]

Do we have any expected timeline ?

@kashifest CR has been released and bump PR is now ready for review #9964

[sbueringer]

Looks like we didn't add 1.29 to versions.md on release-1.6 yet (probably intentional).

We should make sure versions.md is updated once https://github.com/kubernetes-sigs/cluster-api/pull/9971 merges on main

(added a sub-task)

[sbueringer]

@chrischdi I think now we only have one sub-task for docs and "Bump golang on release-1.6 and release-1.5 branches"

I would move the latter to a separate issue so we can close the current one, WDYT?

[chrischdi]

Sounds good 👍

[sbueringer]

Moved to https://github.com/kubernetes-sigs/cluster-api/issues/10025

Otherwise we're done. Thx everyone :)

/close

[k8s-ci-robot]

@sbueringer: Closing this issue.

In response to [this](https://github.com/kubernetes-sigs/cluster-api/issues/9578#issuecomment-1900154187): >Moved to https://github.com/kubernetes-sigs/cluster-api/issues/10025 > >Otherwise we're done. Thx everyone :) > >/close Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.