Launch tasks similarly for both Fenzo and K8s Scheduler pools - Githubissues

twosigma / Cook

Fair job scheduler on Kubernetes and Mesos for batch workloads and Spark

Apache License 2.0

337 stars 63 forks source link

Launch tasks similarly for both Fenzo and K8s Scheduler pools #2161

Closed ahaysx closed 2 years ago

ahaysx commented 2 years ago

Changes proposed in this PR

Refactor launch-tasks-for-cluster to handle launching for Fenzo and Kubernetes Scheduler pools. This required some movement of metrics handling and a boolean gate for Fenzo pools.
Refactors Kubernetes Scheduler pools to use this function for launching, by providing its incomplete match objects (just the metadata) and db transactions.

Why are we making these changes?

Follow-up feedback from #2150