[Feature] Support tolerations when deploying Image Cleaner (Eraser)

[amsoedal]

Is your feature request related to a problem? Please describe. I love having Image Cleaner enabled on our clusters as it prevents vulnerable images from building up. One issue that I'm running into is that I have to manually edit the deployment to be able to add in tolerations. Sometimes (possibly after node reboots, not exactly sure how to repro) the deployment resets and my tolerations are gone. I discover weeks later that the eraser pods have been unable to be scheduled:

Describe the solution you'd like I'd like to be able to add tolerations to the deployment in such a way that they won't be reset. Perhaps an azure CLI experience like: az aks update -g MyResourceGroup -n MyManagedCluster \ --image-cleaner-tolerations key=layer operator=Equal value=api effect=NoSchedule

[fplantinga-guida]

We have the same issue as we run AKS with 3 tainted nodepools. The imagecleaner deployment should at least have the CriticalAddonsOnly toleration

0/3 nodes are available: 1 node(s) had untolerated taint {CriticalAddonsOnly: true}, 1 node(s) had untolerated taint {tenant1: true}, 1 node(s) had untolerated taint {tenant2: true}. preemption: 0/3 nodes are available: 3 Preemption is not helpful for scheduling.

[microsoft-github-policy-service[bot]]

Action required from @Azure/aks-pm

[microsoft-github-policy-service[bot]]

Issue needing attention of @Azure/aks-leads

[microsoft-github-policy-service[bot]]

Issue needing attention of @Azure/aks-leads

[microsoft-github-policy-service[bot]]

Issue needing attention of @Azure/aks-leads

[microsoft-github-policy-service[bot]]

Issue needing attention of @Azure/aks-leads

[microsoft-github-policy-service[bot]]

Issue needing attention of @Azure/aks-leads

[microsoft-github-policy-service[bot]]

Issue needing attention of @Azure/aks-leads

[microsoft-github-policy-service[bot]]

Issue needing attention of @Azure/aks-leads

[microsoft-github-policy-service[bot]]

Issue needing attention of @Azure/aks-leads

[microsoft-github-policy-service[bot]]

Issue needing attention of @Azure/aks-leads

[microsoft-github-policy-service[bot]]

Issue needing attention of @Azure/aks-leads

[microsoft-github-policy-service[bot]]

Issue needing attention of @Azure/aks-leads

[microsoft-github-policy-service[bot]]

Issue needing attention of @Azure/aks-leads

[microsoft-github-policy-service[bot]]

Issue needing attention of @Azure/aks-leads

[microsoft-github-policy-service[bot]]

Issue needing attention of @Azure/aks-leads

[microsoft-github-policy-service[bot]]

Issue needing attention of @Azure/aks-leads

[microsoft-github-policy-service[bot]]

Issue needing attention of @Azure/aks-leads

[microsoft-github-policy-service[bot]]

Issue needing attention of @Azure/aks-leads

[microsoft-github-policy-service[bot]]

Issue needing attention of @Azure/aks-leads