Delazy some of the list generation; do it at rank

twosigma / Cook

Fair job scheduler on Kubernetes and Mesos for batch workloads and Spark

Apache License 2.0

337 stars 63 forks source link

Delazy some of the list generation; do it at rank #2145

Closed scrosby closed 2 years ago

scrosby commented 2 years ago

Changes proposed in this PR

Delazy some of the list generation; do it at rank time to overlap generation with launch.

Why are we making these changes?

This slows down rank time by forcing the lazy list then, but should speed up the first one or two match cycles, and help parallelize the delazy'ing the ranked jobs and launching.