palantir / k8s-spark-scheduler

A Kubernetes Scheduler Extender to provide gang scheduling support for Spark on Kubernetes
Apache License 2.0
175 stars 43 forks source link

When scheduling dynamic executors, order nodes by usage instead of picking a node at random #230

Closed Alexis-D closed 1 year ago

Alexis-D commented 1 year ago

The current logic just picks a random node on which the executor will fit:

https://github.com/palantir/k8s-spark-scheduler/blob/36236cf049a5c730afa5e7107eb3ef0fb624cc35/internal/extender/resource.go#L435-L442

This PR simply tries to fit the new executor in the 'fullest' node.

svc-autorelease commented 1 year ago

Released 0.43.0