[Jobs] Speed up the time for managed jobs to be scheduled

[Michaelvll]

When submitting a lot of managed jobs, the managed jobs will be scheduled slower than the submission speed. This is on a jobs controller with 16 core CPUs.

import subprocess
from multiprocessing.pool import ThreadPool

def run_task(task):
    print(f'Running task {task}')
    subprocess.run(
        f'sky jobs launch -n job-{task} -dy --fast --cloud aws -t t3.medium --use-spot "echo hi {task}; sleep 3600"',
        shell=True
    )

with ThreadPool(8) as pool:
    pool.map(run_task, range(1000))

The gap between jobs being scheduled can be more than 20 seconds.

Version & Commit info:

• sky -v: PLEASE_FILL_IN
• sky -c: PLEASE_FILL_IN