search

googleforgames / agones

Dedicated Game Server Hosting and Scaling for Multiplayer Games on Kubernetes
https://agones.dev
Apache License 2.0
6.09k stars 810 forks source link

Upgrade to Kubernetes 1.16 #1649

Closed roberthbailey closed 4 years ago

roberthbailey commented 4 years ago

As discussed on the monthly community meeting today, it's time to think about when we should upgrade our official support to Kubernetes 1.16.

It looks like we might be on track to do it for the Agones 1.8 release.

List of items to do for upgrading to 1.16 (this is copied from the 1.15 issue and may need to be updated):

roberthbailey commented 4 years ago

The Kubernetes v1.16 Release Notes have some points that we should keep in mind for this upgrade:

roberthbailey commented 4 years ago

We should switch to 1.16 after this next release is cut, as Azure is about to put 1.18 into GA (estimated date is Aug 20th) at which point they will drop official support for 1.15.

markmandel commented 4 years ago

Working up updating the dev gke and e2e clusters to 1.16 -- which is now powered by the GKE terraform module, so taking care of several tasks in one go.

thisisnotapril commented 4 years ago

Just heads-up that the final 1.16 patch is scheduled for 9/2

roberthbailey commented 4 years ago

Ugh. Always on the tail end.....

That being said, GKE's release notes for today just remove support for new clusters at 1.14 and are upgrading 1.14 -> 1.15 in the stable channel while the regular channel is being moved from 1.15 -> 1.16. So we are right in line with where GKE is at in terms of k8s version support.

aLekSer commented 4 years ago

I would add a PR for AKS and EKS soon.

aLekSer commented 4 years ago

Adding some notes from the Agones Community Meeting, mentioned by @roberthbailey :

1.16 has a few api changes to be aware of - things graduating from beta -> ga, older beta apis being dropped We reference beta APIs in our go code: https://github.com/googleforgames/agones/search?l=Go&q=v1beta1 A good first step would be to move the resources in the Helm chart (like CRDs) from betav1 to v1 E.g. Many beta APIs are no longer served: use apps/v1, networking.k8s.io/v1, policy/v1beta1, networking.k8s.io/v1beta1, scheduling.k8s.io/v1 - we should explore and see if we have any.

https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.16.md#deprecations-and-removals

The apiextensions.k8s.io/v1beta1 version of CustomResourceDefinition is deprecated and will no longer be served in v1.19. Use apiextensions.k8s.io/v1 instead.

roberthbailey commented 4 years ago

Most of this is in this issue above.

The important things to catch when upgrading to 1.16 are to make sure that we aren't relying on APIs that are no longer being served (like scheduling.k8s.io/v1beta1).

For APIs that graduated to v1 in 1.16, if we immediately adopt the v1 API then we will prevent Agones from being installed on older versions of k8s. I'd suggest that we wait to take advantage of the newly promoted APIs until 1.17 (or even 1.18) to give folks a longer window to upgrade (even though as a project we won't be testing those versions any longer).

markmandel commented 4 years ago

Working on upgrading client-go :+1:

markmandel commented 4 years ago

Adding to this list:

markmandel commented 4 years ago

A reminder - RC is next week, so probably makes sense to prioritise work for this ticket over others, so we can get support for 1.16 out the door.

markmandel commented 4 years ago

Did a review - looks like it's just the EKS/AKS stuff that needs updating. @aLekSer are you on that?

And to confirm:

Move API references from beta to GA. For instance, scheduling.k8s.io/v1beta1 to scheduling.k8s.io/v1. See https://github.com/googleforgames/agones/search?l=Go&q=v1beta1

We're waiting on 1.17 to do this (@roberthbailey ) ?

roberthbailey commented 4 years ago

For things that just graduated to GA, we are waiting until 1.17 to migrate our configs (see https://github.com/googleforgames/agones/issues/1799).

For things where beta was dropped in 1.16 (like scheduling.k8s.io/v1beta1) we need to move now. But I did a search and only found one instance which was updated already in https://github.com/googleforgames/agones/pull/1714 so that task is complete.

aLekSer commented 4 years ago

AKS is ready, regarding EKS I have a draft PR working on the fix.

markmandel commented 4 years ago

For things where beta was dropped in 1.16 (like scheduling.k8s.io/v1beta1) we need to move now. But I did a search and only found one instance which was updated already in #1714 so that task is complete.

Sweet. Ticked the box.

aLekSer commented 4 years ago

All ticks are in place. Just want to add that we need to add check Metrics checkbox as some names might be changed, there is a section about metrics: https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.16.md#metrics-changes I look through our metrics, definitely that's not the case right now.

roberthbailey commented 4 years ago

Thanks @aLekSer. I've created https://github.com/googleforgames/agones/issues/1824 for the 1.17 upgrade so that we can start capturing any additional steps there. Please feel free to edit that issue to add any steps that you know are missing.

markmandel commented 4 years ago

Awesome work. Any objections to closing this issue then?

roberthbailey commented 4 years ago

I think we are all set.

markmandel commented 4 years ago

CLOSIING!