Add a new extension point EvictPlugin to descheduling framework

[songtao98]

Is your feature request related to a problem? Please describe.

The current descheduling framework defines an EvictorPlugin interface, which is intended to customize the actions of the evictor, such as filtering which pods can be evicted. The descheduler can only evict pods by haveing PodEvictor call the Eviction API. However, there is a need to customize how descheduler actually evict pods. For instance, KEP-1397 discusses about introducing the Evacuation API into descheduler, which would support pod eviction requests that are not completed right away.

Describe the solution you'd like

A new extension point interface EvictPlugin could help to allow users config how descheduler really evict pods.

type EvictPlugin interface {
    Plugin

    // Evict evicts a pod (no pre-check performed)
    Evict(ctx context.Context, pod *corev1.Pod, evictOptions EvictOptions) bool
}

PodEvictor could be replaced by custom evictors through DeschedulerPolicy by a new param evict just like deschedule, balance, filter and preEvictionFilter.

apiVersion: "descheduler/v1alpha2"
kind: "DeschedulerPolicy"
[...]
profiles:
  - name: descheduler-with-custom-evictor
    plugins:
      deschedule:
        enabled:
          - ...
      evict:
        enabled:
          - CustomEvictor
      [...]

Describe alternatives you've considered

Add a new configuration argument into PodEvictor, allowing users to decide whether to use Evacuation API or Eviction API. But there might be other specific logic that users want to customize in their own Evictors.

What version of descheduler are you using?

descheduler version: 0.29.0

Additional context

[songtao98]

A detailed KEP to be written