skypilot-org / skypilot

SkyPilot: Run AI and batch jobs on any infra (Kubernetes or 12+ clouds). Get unified execution, cost savings, and high GPU availability via a simple interface.
https://skypilot.readthedocs.io
Apache License 2.0
6.81k stars 513 forks source link

[Core] Speed up job scheduling speed on unmanaged jobs #4295

Closed Michaelvll closed 6 days ago

Michaelvll commented 2 weeks ago

Although we recently speed up the unmanaged job scheduling speed in #4264, it still takes 2-3 seconds to schedule job one by one. We should speed this up, especially for large-scale job submission, this can cause a significant delay.

Version & Commit info: