kubernetes-sigs / descheduler

Descheduler for Kubernetes
https://sigs.k8s.io/descheduler
Apache License 2.0
4.51k stars 672 forks source link

feature: support setting gracePeriodSeconds in DeschedulerPolicy #1537

Open googs1025 opened 1 month ago

googs1025 commented 1 month ago

Is your feature request related to a problem? Please describe.

GracePeriodSeconds is important for setting up evictions because it ensures that Pods have enough time to perform necessary cleanup operations before being deleted, thereby ensuring data consistency and service availability. Depending on application requirements, setting GracePeriodSeconds appropriately can improve system stability and reliability.

type DeschedulerPolicy struct {
    metav1.TypeMeta

    // Profiles
    Profiles []DeschedulerProfile
    ...
    // GracePeriodSeconds The duration in seconds before the object should be deleted. Value must be non-negative integer.
    // The value zero indicates delete immediately. If this value is nil, the default grace period for the
    // specified type will be used.
    // Defaults to a per object value if not specified. zero means delete immediately.
    GracePeriodSeconds int64
}

Describe the solution you'd like

https://github.com/kubernetes-sigs/descheduler/blob/ef0c2c1c47c6aa6afb0d08f4a2488d7df004c2f7/pkg/descheduler/evictions/evictions.go#L199-L222

Describe alternatives you've considered

What version of descheduler are you using?

descheduler version:

Additional context

googs1025 commented 1 month ago

/assign

googs1025 commented 1 month ago

@damemi @ingvagabund @a7i Do we need this configuration feature?

a7i commented 1 month ago

We've never had a use-case for it, typically eviction needs to happen immediately.

What is your use-case for this?

googs1025 commented 1 month ago

@a7i thanks for quick reply Within our internal cluster, we have some long-running batch jobs or other business processes. However, when employing the descheduler for evictions (for instance, when the cluster reaches high utilization levels), the business holders wish to have the ability to set gracePeriodSeconds during evictions. This capability ensures that custom cleanup procedures have sufficient time to execute, thereby preventing issues arising from abrupt interruptions.

Furthermore, our SREs, during manual node maintenance using kubectl drain, typically include a reasonable --grace-period flag. We aim to have the same functionality available when utilizing the descheduler for rescheduling, maintaining consistency in the approach across both evictions and node maintenance operations.

ingvagabund commented 1 month ago

https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.31/#create-eviction-pod-v1-core eviction API does not currently provides means to specify gracePeriodSeconds.

ingvagabund commented 1 month ago

the user can set it from the configuration file and pass it to the deleteOptions of evict.

@googs1025 I wonder how this works. Have you had a chance to implement it and see whether it works?

googs1025 commented 1 month ago

https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.31/#create-eviction-pod-v1-core eviction API does not currently provides means to specify gracePeriodSeconds.

We currently use client.PolicyV1().Evictions(eviction.Namespace).Evict(ctx, eviction) to perform evictions, which contains the DeleteOptions option. Simply passing in the configuration should work.

https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.31/#eviction-v1-policy

googs1025 commented 1 month ago

the user can set it from the configuration file and pass it to the deleteOptions of evict.

@googs1025 I wonder how this works. Have you had a chance to implement it and see whether it works?

This is a WIP pr: https://github.com/kubernetes-sigs/descheduler/pull/1538