search

kr8s-org / kr8s

A batteries-included Python client library for Kubernetes that feels familiar for folks who already know how to use kubectl
https://kr8s.org
BSD 3-Clause "New" or "Revised" License
793 stars 40 forks source link

Cloud Kubernetes Extended Version Support #467

Open jacobtomlinson opened 1 month ago

jacobtomlinson commented 1 month ago

Currently kr8s maintains support for Kubernetes versions that are actively supported by the OSS Kubernetes community. We have a script which tracks the currently supported Kubernetes versions (via endoflife.date, a community EOL date tracker) and updates the CI automatically to add new versions and remove old unsupported versions.

However, many users use cloud managed Kubernetes deployments from services like Amazon EKS, Google Cloud GKE and Azure AKS. The managed services maintain different EOL dates, and some offer extended support.

It might make sense to extend the kr8s Kubernetes supported versions to include versions commonly still supported by the managed Kubernetes services.

Clouds

Google GKE

Google maintain 14 months of support for new Kubernetes versions. This is two months longer than the open source projects.

Azure AKS

Azure maintains the latest two versions of Kubernetes. When a new version is added the third oldest is supported for 30 more days and then removed. New versions are added on an unknown schedule.

Azure is also adding LTS support for some Kubernetes versions where they add an extra year of support.

Amazon EKS

AWS support is similar to the standard AKS support and extended GKE support. However they provide the most generous maintenance support where all versions are supported for an additional 12 months beyond their standard EOL.

Options

Here are a few idea of things we could choose to do.

Option 1: Do nothing

We could continue to keep support with OSS Kubernetes and remove old versions once they leave active support.

Option 2: Follow Cloud Active Support

We could follow the active support dates of the cloud vendors. If we take Kubernetes 1.27 as an example OSS Kubernetes was the first to drop support on April 28th 2024 and Azure AKS was the last to drop support on July 31st 2024. This would extend our support by roughly four months which would add one additional CI job to every run.

Option 3: Follow Cloud Maintenance Support

We could also follow the maintenance support of the clouds to ensure that all Kubernetes versions with any level of support are still supported in kr8s. We add new versions here as soon as the OSS release happens, but the clouds can take up to nine months to add a new version. If they then support that version for just over two years we could end up needing to maintain support for nearly three years.

Using Kubernetes 1.22 as an example the initial OSS release happened on August 4th 2021 and is supported by EKS extended support until September 1st 2024. On average we would need to support older Kubernetes versions for an additional 14 months which would add roughly six CI jobs to every run, this would nearly double the number of CI jobs.

jacobtomlinson commented 1 month ago

I feel like Option 2 is the most pragmatic. We can support all Kubernetes versions which are actively supported by the cloud vendors by adding only one CI job per workflow run and delaying our ability to deprecate old things by four months.

We could also consider finding a way to make it easier for users to pin to older versions of kr8s. Currently if a user is still on Kubernetes 1.27 they would need to find out which version of kr8s was the last to support that version of Kubernetes and pin to there. The only way to do that today is to go back through the git blame to find out when support was removed and then find the release that includes that PR. Which isn't very helpful.

One solution could be to publish a meta package to PyPI that follows the Kubernetes versioning exactly. We could add some automation so that every time a new Kubernetes tag is created we create the same tag on the meta package. That way we can add the meta package as a dependency to kr8s and users could pin the meta package to correctly resolve the version of kr8s that they need.