Performance improvement

volcano-sh / volcano

A Cloud Native Batch System (Project under CNCF)

https://volcano.sh

Apache License 2.0

3.9k stars 902 forks source link

Performance improvement #3502

Open william-wang opened 3 weeks ago

william-wang commented 3 weeks ago

What would you like to be added:

Optimize the controllers and schedulers components to improve the performence from end to end for high performance workload.

Why is this needed:

More and more workloads are being migrated to Kubernetes on Volcano, Some workloads have higher requirements for high throughput and low latency, e.g bilibili requires best qps for vedio transcoding from job created to pod running.

sub-tasks

[ ] Controller
[ ] Scheduler

Monokaix commented 3 weeks ago

Controller related:

[x] https://github.com/volcano-sh/volcano/issues/3490
[ ] https://github.com/volcano-sh/volcano/issues/3485
[x] https://github.com/volcano-sh/volcano/issues/3484
[x] https://github.com/volcano-sh/volcano/issues/3488

lowang-bh commented 3 weeks ago

Scheduler related:

allocate action:

[ ] It's better to check and skip allocation for job when it is poped #3047
[ ] cache feasible nodes for a job's tasks #3492
[ ] record the norminated node for pod and consider it first in next cycle #3491

preempt action:

[ ] filterout those nodes which are not helpful for preemptation #3071