search

stanford-futuredata / gavel

Code for "Heterogenity-Aware Cluster Scheduling Policies for Deep Learning Workloads", which appeared at OSDI 2020
MIT License
125 stars 31 forks source link

Running scheduler_tests.py #241

Open nayakajay opened 2 years ago

nayakajay commented 2 years ago

I wanted to run the test scheduler_tests.py. I believe, for a given trace, this test will give me the schedule in a file /tmp/simple.output.

The traces used seems to be missing. I can use a different trace, but it does not have an "expected" output file for those. Also, the relevant file (run_scheduler_with_trace.py) is not prepared to take the arguments passed by the test.

deepakn94 commented 2 years ago

It looks like that test file is out-of-date :/ What arguments are not accepted by the driver?

nayakajay commented 2 years ago

There are 3 arguments that are not supported. --simulate --worker_types -l

The simulate argument can be handled as it can be read from command line and passed to the Scheduler's constructor. However, the other two are not expected arguments for the Scheduler class. I am not sure where they are to be passed or how to handle them.

deepakn94 commented 2 years ago

Actually, you want to use simulate_scheduler_with_trace.py: https://github.com/stanford-futuredata/gavel/blob/master/scheduler/scripts/drivers/simulate_scheduler_with_trace.py.

You can get rid of the --simulate argument then, simulate is set to True by default in this driver file. --worker_types is replaced by --cluster_spec. I'm not sure what -l did before, but you can probably omit it.