palantir / k8s-spark-scheduler

A Kubernetes Scheduler Extender to provide gang scheduling support for Spark on Kubernetes
Apache License 2.0
175 stars 43 forks source link

Deduplicate driver pod demands #237

Closed Chrisbattarbee closed 1 year ago

Chrisbattarbee commented 1 year ago

Before this PR

Before this PR we would make demands for the entire application: The driver pod + all executor pods. This was a problem because the driver pod also existed in the cluster as well as the demand which was needed to provide scale up signal in a particular AZ.

After this PR

This PR adds the driver pod to PodNamesByNamespace which allows scaling infrastructure to to deduplicate driver pods and their associated demands

==COMMIT_MSG== Deduplicate driver pod demands ==COMMIT_MSG==

svc-autorelease commented 1 year ago

Released 0.48.0