mle-infrastructure / mle-toolbox

Lightweight Tool to Manage Distributed ML Experiments 🛠
https://mle-infrastructure.github.io/mle_toolbox/toolbox/
MIT License
3 stars 1 forks source link

Multi-Config Experiments with Sequential Batches #60

Closed RobertTLange closed 3 years ago

RobertTLange commented 3 years ago

multi_config experiments currently schedule all jobs at once. That makes sense since there is no sequentiality and the order of evaluation doesn't matter. Still it would be nice to give the user more control over the resources demanded. Ideally, we would give an optional argument to the multi_experiment_args in the experiment .yaml, which states max_number_of_running_jobs at a time or something like that.

This connects with #8 & #9 and should probably addressed together.