mle-infrastructure / mle-toolbox

Lightweight Tool to Manage Distributed ML Experiments 🛠
https://mle-infrastructure.github.io/mle_toolbox/toolbox/
MIT License
2 stars 0 forks source link

`MLELauncher` - Launch experiment from notebook #91

Open RobertTLange opened 2 years ago

RobertTLange commented 2 years ago

I would like to have a simple object-oriented API that allows you to launch an experiment from a notebook. E.g. via something like

from mle_toolbox import MLELauncher

experiment_yaml = "base_config.yaml"
purpose = "some purpose""
resource = "slurm""

# Setup launcher - loads configs, etc.
experiment = MLELauncher()
# Startup experiment - copy files to SSH destination
experiment.run(experiment_yaml, purpose, resource)

Once the experiment is completed we can then do further post-processing, e.g.:

# scp over files
experiment.retrieve()

# create a report
experiment.report()