Propose a more intuitive queue priority

[Monokaix]

What would you like to be added:

Add a field of queue spec to let user set queue priority directly, then schduler will schdule jobs and sort queues by this priority .

Why is this needed:

Today volcano use queue's share value to distinguish queue's priority, and a lower share can get a higher priority and will be scheduled first, related doc: https://volcano.sh/en/docs/v1-8-2/plugins/#drf, but in most cases, users want set queue priority directly, which intuitively exposes queue priorities to users, and also is convenient to debug. We can compare queue's priority first and then fall back to queue's share value if queues have same priority.

[Monokaix]

/assign

[TaiPark]

Queue Priority

Motivation

The current queue prioritization in Volcano is based on the principle that a lower share value indicates higher priority for resource allocation during scheduling.

However, based on user feedback and practical scenarios, there is a strong preference for a more straightforward approach to establishing queue priorities. Explicitly assigning priorities provides users with clear and immediate control over the order in which their queues are serviced.

While Volcano's current method of using queue shares effectively determines priority, there is a clear demand for a more intuitive and user-friendly approach to setting and managing queue order. Enabling users to directly assign and adjust queue priorities in the specifications would simplify queue management significantly.

Implementation

Data Structure

Add priority to spec of queues.scheduling.volcano.sh. The priority attribute controls the order of queues in all plugins that implement QueueOrderFn.

spec:
    ...
    priority:
        type: number
    ...

The range for priority is from 0 to the maximum limit of int32.

Queue Ordering

Under the premise that queues that are overused will not enter the scheduling process, all QueueOrderFn plugins adhere to the following sorting logic:

• Check queue priority first, queues with higher priority are positioned at the front of the PriorityQueue and are popped first.
• When queue priority values are identical, the logic reverts to the current method of comparing queue shares.

[Monokaix]

Queue Priority

Motivation

The current queue prioritization in Volcano is based on the principle that a lower share value indicates higher priority for resource allocation during scheduling.

However, based on user feedback and practical scenarios, there is a strong preference for a more straightforward approach to establishing queue priorities. Explicitly assigning priorities provides users with clear and immediate control over the order in which their queues are serviced.

While Volcano's current method of using queue shares effectively determines priority, there is a clear demand for a more intuitive and user-friendly approach to setting and managing queue order. Enabling users to directly assign and adjust queue priorities in the specifications would simplify queue management significantly.

Implementation

Data Structure

Add priority to spec of queues.scheduling.volcano.sh. The priority attribute controls the order of queues in all plugins that implement QueueOrderFn.

spec:
    ...
    priority:
        type: number
    ...

The range for priority is from 0 to the maximum limit of int32.

Queue Ordering

Under the premise that queues that are overused will not enter the scheduling process, all QueueOrderFn plugins adhere to the following sorting logic:

• Check queue priority first, queues with higher priority are positioned at the front of the PriorityQueue and are popped first.
• When queue priority values are identical, the logic reverts to the current method of comparing queue shares.

Please also descibe the behavior of scheduler in queue reclaim case, and submit a design pr individually: )

[Monokaix]

/assign @TaiPark

[Monokaix]

/close

[volcano-sh-bot]

@Monokaix: Closing this issue.

In response to [this](https://github.com/volcano-sh/volcano/issues/3542#issuecomment-2345184189): >/close Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.