search

mle-infrastructure / mle-toolbox

Lightweight Tool to Manage Distributed ML Experiments 🛠
https://mle-infrastructure.github.io/mle_toolbox/toolbox/
MIT License
3 stars 1 forks source link

More general options for subcommands #70

Open RobertTLange opened 3 years ago

RobertTLange commented 3 years ago

I would like to allow users to specify more details directly via the command line and not only via the config .yaml. For mle run this could for example include:

num_seeds: <num_seeds>
num_logical_cores: <num_logical_cores>
num_gpus: <num_gpus>

These (and others) should all be added to run_build_subparser:

https://github.com/RobertTLange/mle-toolbox/blob/5c6a475d1b5d01e6e4f947a3cb426489c7ed9427/mle_toolbox/toolbox.py#L52-L91

We can then simply overwrite the arguments in the config using load_yaml_config.

https://github.com/RobertTLange/mle-toolbox/blob/5c6a475d1b5d01e6e4f947a3cb426489c7ed9427/mle_toolbox/utils/core_files_load.py#L44-L56

RobertTLange commented 2 years ago

Add an experiment mle run-remote command that allows you to launch a job from your local machine on Slurm, GridEngine, SSH, GCloud.

RobertTLange commented 2 years ago

And also add --report option to automatically also create a report.