Support new topologySpread scheduling constraints

[tzneal]

Description

Observed Behavior:

Karpenter doesn't support:

• matchLabelKeys
• nodeAffinityPolicy
• nodeTaintsPolicy

These fields were introduced into beta in 1.27.

Expected Behavior:

Reproduction Steps (Please include YAML):

Versions:

• Chart Version:

• Kubernetes Version (kubectl version):

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request

• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request

• If you are interested in working on this issue or have submitted a pull request, please leave a comment

[sadath-12]

I'll work on this issue

[jmdeal]

/assign

[jmdeal]

/remove-help

[ilkinmammadzada]

What is current behaviour of the karpenter for nodeAffinityPolicy?

[jmdeal]

Currently Karpenter doesn't recognize either nodeAffinityPolicy, so it will always be treated as honor. Originally the linked PR was blocked on some 1.29 features and I haven't had the time to do the performance testing since we got unblocked on 1.29 testing. Those features have since been scoped out but the linked PR will add support for nodeAffinityPolicy, nodeTaintPolicy, and matchLabelKeys.

[dominik-dezordo-vc]

Hey everyone! Thanks for putting in the effort to support these new constraints! We just hit this exact issue in our clusters, where we are using matchLabelKeys together with topologySpreadConstraints to spread our pods across AZs.

I was wondering if you currently have a timeline for getting this PR merged? It seems the code is already ready for some time and it is just waiting for a rebase/performance test?

[jmdeal]

This has been backlogged for me for a little while, but I should have some bandwidth to get this wrapped out within the next week. Like you said, it really should only be performance testing and a rebase at this point that's left.

[dominik-dezordo-vc]

Amazing, thanks for the fast response. Really appreciate the work you are doing with karpenter!

[pznamensky]

We are also really awaiting for matchLabelKeys as now topologySpread is applied to all versions of a deployment which sometimes works not as optimal as we would like.

[abrazouski]

Hello everyone In my company, we heavily rely on the matchLabelKeys setting and we've also encountered similar issues when new pods of the rollout were in the Pending because Karpenter didn't respect the matchLabelKeys and made the calculations taking into account pods of the previous revision. It caused an incident during the night preventing us from applying hotfixes to the production workloads.

Could you please share the ETA of this feature?

[hamishforbes]

Hi, the lack of matchLabelKeys support, in particular, is also causing problems for us.

Is there anything we can do to help, test a custom build of Karpenter that supports this feature or something?