mle-infrastructure / mle-toolbox

Lightweight Tool to Manage Distributed ML Experiments 🛠
https://mle-infrastructure.github.io/mle_toolbox/toolbox/
MIT License
3 stars 1 forks source link

GridSearch for multiple base configurations #87

Closed RobertTLange closed 2 years ago

RobertTLange commented 3 years ago

Make it possible to run the same grid search for multiple base configurations within a single experiment. E.g. by stating them in the experiment config:


# Meta Arguments: What job? What train .py file? Base config? Where to store?
meta_job_args:
    project_name: "drl-lth"
    experiment_type: "hyperparameter-search"
    base_train_fname: "cluster_exec_mle.sh"
    base_train_config: 
        - "configs/train/bullet/config_1.json"
        - "configs/train/bullet/config_2.json"
    experiment_dir: "runs/bullet/calibrate/v3_2/"
    report_generation: false

The hyper_log.pkl should in that case have a separate variable that saves the configuration file name. It would also be nice to automatically calculate the number of jobs/batches given a fixed amount of jobs per batch/max to run.