[BUG] Automated OS patch upgrades not patching nodes of deprecated patch versions clusters ( Even though minor version is still supported)

ashwajce commented 1 year ago

Describe the bug

Currently, we are running many clusters on patch version 1.25.5 (Westeurope & France Central), This patch version is currently deprecated & now out of support (the current supported versions are 1.25.6 and 1.25.11). We have subscribed to the automatic_channel_upgrade for node-image. (According to the documentation, setting the cluster auto-upgrade channel to node-image also automatically sets the node OS auto-upgrade channel to NodeImage, as explained here: Auto-upgrade cluster)

As per the NodeImage documentation Auto-upgrade node image, "Node image upgrades will work on patch versions that are deprecated, as long as the minor Kubernetes version is still supported."

The current node OS version is AKSUbuntu-2204gen2containerd-202303.22.0, and it should have been updated to either AKSUbuntu-2204gen2containerd-202309.06.0 or at least to AKSUbuntu-2204gen2containerd-202308.28.0. However, this is not the case. When we raised a ticket with Microsoft, the response was: "Current MC version 1.25.5 is not supported. We will skip agent pool upgrade and try to upgrade the control plane first."

To Reproduce Steps to reproduce the behavior:

1) Create a cluster with version 1.25.5 and set the auto-upgrade channel to node-image. 2) Verify if you are receiving automatic OS updates. 3) Wait until this patch version becomes unsupported. 4) Verify if you are still receiving automated OS patch updates.

Expected behavior Automated OS patch upgrades should patch nodes of deprecated patch versions of the cluster, as long as their minor Kubernetes version is still supported, as stated in the documentation.

Environment:

Kubernetes version 1.25.5

adejongh commented 1 year ago

Perhaps it is related to my issue here: https://github.com/Azure/AKS/issues/3918

I am also unable to get any of these updates to work - we are on a supported patch level (1.26.6) though, but it is an older AKS cluster with "Azure AD authentication with Kubernetes RBAC v1 (Legacy)".