Azure / azure-service-operator

Azure Service Operator allows you to create Azure resources using kubectl
https://azure.github.io/azure-service-operator/
MIT License
741 stars 196 forks source link

Bug: Disabling Automatic Upgrade Do Not Work #4082

Closed RSE132 closed 4 months ago

RSE132 commented 4 months ago

v2.7.0

NAME                                      READY   UP-TO-DATE   AVAILABLE   AGE    CONTAINERS   IMAGES                                              SELECTOR
azureserviceoperator-controller-manager   1/1     1            1           5d4h   manager    mcr.microsoft.com/k8s/azureserviceoperator:v2.7.0   control-plane=controller-manager

Describe the bug According to the PSRule for Azure AKS auto-upgrade channel, the following release channels are available:

none - Disables auto-upgrades. The default setting. patch - Automatically upgrade to the latest supported patch version of the current minor version. stable - Automatically upgrade to the latest supported patch release of the recommended minor version. This is N-1 of the current AKS non-preview minor version. rapid - Automatically upgrade to the latest supported patch of the latest support minor version. node-image - Automatically upgrade to the latest node image version. Normally upgraded weekly.

We want to keep it to none, hence used the spec as described in ASO V2 sample with upgradeChannel: none - https://github.com/Azure/azure-service-operator/blob/f288e217c85bb04b7d5e60d82218f6223737aae7/v2/samples/dataprotection/v1api20231101/refs/v1api20231001_managedcluster.yaml#L20

But each time cluster is created, we could see in the azure portal that the upgradeChannel value is set to node-image although the applied manifest/yaml shows upgradeChannel: none

Screenshot from Azure Portal

image

Applied manifest snippet

  "autoUpgradeProfile": {
    "upgradeChannel": "none"
  }, 

To Reproduce Steps to reproduce the behavior:

apiVersion: containerservice.azure.com/v1api20231001
kind: ManagedCluster
metadata:
  annotations:
    serviceoperator.azure.com/credential-from: landingzone-aso-credential
    serviceoperator.azure.com/operator-namespace: azureserviceoperator-system
    serviceoperator.azure.com/reconcile-policy: manage
  name: private-sandbox-cluster
  namespace: landing-zone
spec:
# ControlPlane Kubernetes Version
  kubernetesVersion: "1.28.9"
  autoUpgradeProfile:
    upgradeChannel: none
# Resourcegroup
  owner:
    name: private-sandbox-cluster-rgpazewdaks01sandbox-test-aso                   
# Location
  location: westeurope                                    
# Enable OIDC
  oidcIssuerProfile:                                      
    enabled: true
# DNS Prefix
  dnsPrefix: private-sandbox-cluster      
# Private Cluster
  apiServerAccessProfile:
    enablePrivateCluster: true
    privateDNSZone: None              
# Identity type
  identity:                                               
    type: SystemAssigned
# Serviceprinciple Profile
  servicePrincipalProfile:
    clientId: xxxxx-xxxxx-xxxxx-xxxxx-xxxxxxx
    secret:
      key: clientSecret
      name: landingzone-sp-secret
# AKS Support Tier SLA
  sku:
    name: "Base"                                                  
    tier: "Standard"        
# Keep Local kubernetes Auth                               
  disableLocalAccounts: false                             
# AAD Integration      
  aadProfile:    
    managed: true                                         
    adminGroupObjectIDs:                                  # AAD security Group
      - xxxxx-xxxxxx-xxxxxx-xxxxxx-xxxxxx
    enableAzureRBAC: false                                  # Enable AAD with k8s RBAC. Set to false
  agentPoolProfiles:
    - name: agentpool
      count: 1
      vmSize: Standard_D2s_v3
      osType: Linux
      osSKU: Ubuntu
      mode: System

Expected behavior It is expected that upgradeChannel: none also reflects in Azure Portal, otherwise the k8s node update takes place automatically

matthchr commented 4 months ago

There are two fields on the autoUpgradeProfile

One is nodeOSUpgradeChannel and one is upgradeChannel: image

I think you've set the upgradeChannel one only, but not the nodeOSUpgradeChannel. You can confirm this in the portal by looking at the raw JSON of the resource: image

My expectation based on your AKS YAML would be that the portal JSON will show upgradeChannel is none but nodeUSUpgradeChannel is not.

Somewhat related note: it's not recommended to disable node patching unless you're doing patching yourselves.

RSE132 commented 4 months ago

@matthchr

Thanks for your help. It setting nodeUSUpgradeChannel:none worked out.