Closed julie-ng closed 2 years ago
Azure was smart enough to recognize it as a system node based on naming 😅 because I see kubernetes.azure.com/mode=system
$ k describe no aks-system-19810993-vmss000000
Name: aks-system-19810993-vmss000000
Roles: agent
Labels: agentpool=system
beta.kubernetes.io/arch=amd64
beta.kubernetes.io/instance-type=Standard_B2ms
beta.kubernetes.io/os=linux
failure-domain.beta.kubernetes.io/region=norwayeast
failure-domain.beta.kubernetes.io/zone=0
kubernetes.azure.com/agentpool=system
kubernetes.azure.com/cluster=cloudkube-dev-i7iw-managed-rg
kubernetes.azure.com/mode=system
kubernetes.azure.com/node-image-version=AKSUbuntu-1804gen2containerd-2022.03.20
kubernetes.azure.com/os-sku=Ubuntu
kubernetes.azure.com/role=agent
kubernetes.azure.com/storageprofile=managed
kubernetes.azure.com/storagetier=Premium_LRS
kubernetes.io/arch=amd64
kubernetes.io/hostname=aks-system-19810993-vmss000000
kubernetes.io/os=linux
kubernetes.io/role=agent
node-role.kubernetes.io/agent=
node.kubernetes.io/instance-type=Standard_B2ms
storageprofile=managed
storagetier=Premium_LRS
topology.disk.csi.azure.com/zone=
topology.kubernetes.io/region=norwayeast
topology.kubernetes.io/zone=0
workloadType=system
Annotations: csi.volume.kubernetes.io/nodeid:
{"disk.csi.azure.com":"aks-system-19810993-vmss000000","file.csi.azure.com":"aks-system-19810993-vmss000000","secrets-store.csi.k8s.io":"a...
node.alpha.kubernetes.io/ttl: 0
volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp: Thu, 31 Mar 2022 22:24:37 +0200
Nevermind - no taints 😓
CreationTimestamp: Thu, 31 Mar 2022 22:24:37 +0200
Taints: <none>
Unschedulable: false
TIL 😒
Adding taints, labels, or tags to nodes should be done for the entire node pool using az aks nodepool. Applying taints, labels, or tags to individual nodes in a node pool using kubectl is not recommended.
Problem
Currently the
default_node_pool
block is used for the system node pool. But this defaults to user mode.Consequence
AKS has a
mode
property that can be eithersystem
oruser
which beyond semantics adds theCriticalAddonsOnly=true:NoSchedule
taint.Docs Referender: Manage system node pools in Azure Kubernetes Service (AKS) > System and user node pools
Changes required
default_node_pool
block and thekubernetes_cluster_node_pool
resource definitions.