Add CriticalAddonsOnly=true:NoSchedule to az aks create

[aanm]

Describe the request

Right not it seems it's not possible to create a cluster with pre-defined node taints on cluster creation given that --node-taints is not part of of az aks create. A workaround for this would be to create a cluster with 0 nodes, which is also not possible, and then create a node pool with the pre-defined node taints.

Describe the solution you'd like

Either to create a cluster with 0 nodes, for which users would create the node pool with the node taints or allow cluster creation to accept node-taints.

[ghost]

Hi aanm, AKS bot here :wave: Thank you for posting on the AKS Repo, I'll do my best to get a kind human from the AKS team to assist you.

I might be just a bot, but I'm told my suggestions are normally quite good, as such: 1) If this case is urgent, please open a Support Request so that our 24/7 support team may help you faster. 2) Please abide by the AKS repo Guidelines and Code of Conduct. 3) If you're having an issue, could it be described on the AKS Troubleshooting guides or AKS Diagnostics? 4) Make sure your subscribed to the AKS Release Notes to keep up to date with all that's new on AKS. 5) Make sure there isn't a duplicate of this issue already reported. If there is, feel free to close this one and '+1' the existing issue. 6) If you have a question, do take a look at our AKS FAQ. We place the most common ones there!

[ghost]

Triage required from @Azure/aks-pm

[PixelRobots]

I would like to see this also. I am currently having to create a new system node pool with taints and then delete the automatically created one. Wastes a lot of time.

[justindavies]

Hi would you be able to let me know the use case (what taints) you would need on the system nodepool? This will help me to look at these from a feature perspective.

[PixelRobots]

For me it is this: --node-taints CriticalAddonsOnly=true:NoSchedule Which is referenced in the docs. https://docs.microsoft.com/en-us/azure/aks/use-system-pools#add-a-dedicated-system-node-pool-to-an-existing-aks-cluster

[justindavies]

Thanks, I'll get this added to the az cli

[PixelRobots]

Thanks. If you need someone to test, I am happy to help. I have a signed MVP NDA.

[aanm]

Hello @justaniles to me is to prevent application pods from being managed by CNI plugins that are installed by default. If users want to use Cilium, a CNI plugin, there's a concurrency issue where application pods might be managed by Cilium or by the default CNI plugin.

With a node taint, the application pods will not run until Cilium is deployed and removes the taint (node.cilium.io/agent-not-ready=true:NoSchedule) created by the user from the nodes.

Something as:

$ az aks create [...] --node-taints node.cilium.io/agent-not-ready=true:NoSchedule
$ cilium install

A more detailed explanation can be found in here https://github.com/cilium/cilium/issues/16602. Thank you!

[nbusseneau]

Hello @justindavies, I wanted to chime in on this: I feel like the original title allow creation of clusters with 0 nodes or allow node-taints to be set on cluster creation is probably more accurate, as the need is to be able to place arbitrary custom taints on the initial nodepool nodes before they are created, rather than CriticalAddonsOnly=true:NoSchedule specifically.

In any case, I wanted to know if there was any update on that front? Thanks :)

[nbusseneau]

I have just noticed there is already #1402 covering the allow node-taints to be set on cluster creation part, which we seem to have missed previously :D

[PixelRobots]

Az aks create has an option to add taints to the node pool. This can be close.