Azure / AKS

Azure Kubernetes Service
1.92k stars 284 forks source link

Include containerd 1.7.15 into next AKS version #4210

Closed bparentpichette closed 3 weeks ago

bparentpichette commented 1 month ago

Describe scenario

We are using AKS v1.27.9. Since upgrading from version 1.26.10 to 1.27.9, we have been experiencing issues with certain CI/CD jobs executed by our GitLab runner running within the cluster. The problem has been identified, and it is the version of containerd that is causing the issue.

The containerd fix: https://github.com/containerd/containerd/pull/10037 The gitlab-runner issue: https://gitlab.com/gitlab-org/gitlab-runner/-/issues/37446 Confirmation that the containerd update fixes the problem:

Question When will the containerd version 1.7.15 be integrated into AKS?

Thank you

mjnovice commented 1 month ago

We are facing a similar issue. Any ETA on when can this be fixed ?

ankitjain91 commented 1 month ago

guys any update on this issue?

uluzox commented 1 month ago

This would be much appreciated. My team and I are also experiencing stuck CI/CD Jobs with GitLab Runners.

yavor-ivanov-covantis commented 1 month ago

Please prioritize this one!

vnaren23 commented 1 month ago

We are also experiencing the same. Can this fix be expedited please?

UtheMan commented 1 month ago

Containerd v1.7.15 is present in the newest node image version that is now rolling out to all PROD regions. We are expecting it to reach all regions by end of this week. Node image version 202404.09.0 is what has the updated version. You can track the release here: https://releases.aks.azure.com/#tabus (AKS node images tab on the left side). Once the version is in your region, perform node image upgrades for your nodes.

Aaron-ML commented 1 month ago

Containerd v1.7.15 is present in the newest node image version that is now rolling out to all PROD regions. We are expecting it to reach all regions by end of this week. Node image version 202404.09.0 is what has the updated version. You can track the release here: https://releases.aks.azure.com/#tabus (AKS node images tab on the left side). Once the version is in your region, perform node image upgrades for your nodes.

Are we still looking at the EOW for this update to apply to most regions? This created a production issue this week and would like to remediate asap.

UtheMan commented 1 month ago

Containerd v1.7.15 is present in the newest node image version that is now rolling out to all PROD regions. We are expecting it to reach all regions by end of this week. Node image version 202404.09.0 is what has the updated version. You can track the release here: https://releases.aks.azure.com/#tabus (AKS node images tab on the left side). Once the version is in your region, perform node image upgrades for your nodes.

Are we still looking at the EOW for this update to apply to most regions? This created a production issue this week and would like to remediate asap.

Yes, we are still on track to get it to all regions by end of Friday PST time zone.

UtheMan commented 3 weeks ago

This containerd version is now available in all regions, I will close the issue.