kubernetes-sigs / karpenter

Karpenter is a Kubernetes Node Autoscaler built for flexibility, performance, and simplicity.
Apache License 2.0
637 stars 206 forks source link

Make consolidateAfter configurable by disruption reason #1757

Open yagonobre opened 1 month ago

yagonobre commented 1 month ago

Description

What problem are you trying to solve?

We are trying to better control consolidation and avoid pods being moved around too frequently. To achieve this, we started using consolidateAfter. However, this has the side effect of keeping empty nodes in our cluster for longer, leading to additional costs.

It would be ideal if we had the option to configure consolidateAfter based on the disruption reason, allowing us better control over the process. We understand that the Karpenter API is now stable, but what we are proposing is to add an optional consolidateAfter configuration per disruption reason, which would take higher priority than the global setting.

Here is a api suggestion:

spec:
  disruption:
    budgets:
    - nodes: "2%"
      schedule: "0 21 * * *"
      duration: 12h
      reasons:
      - Drifted
    - nodes: "100%"
      consolidateAfter: 15m
      reasons:
      - Empty
    - nodes: "2%"
      reasons:
      - Underutilized
    consolidateAfter: 12h

Her we should have Drifted and Unterutilized Nodes consolidated after 12h but Empty nodes after 15m.

How important is this feature to you?

k8s-ci-robot commented 1 month ago

This issue is currently awaiting triage.

If Karpenter contributors determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.