iterative / example-repos-dev

Source code and generator scripts for example DVC projects
https://dvc.org/doc
21 stars 13 forks source link

experiments: Include model in the experimetns inside notebook #171

Closed daavoo closed 1 year ago

daavoo commented 1 year ago

In the current notebook, we are creating multiple experiments inside a cell by using save_dvc_exp=True inside a for loop:

for base_lr in [0.001, 0.005, 0.01]:
    with Live(str(ROOT / "results" / "train"), save_dvc_exp=True) as live:
        live.log_param("train_arch", train_arch)
        fine_tune_args = {
            'epochs': 8,
            'base_lr': base_lr
        }
        live.log_params(fine_tune_args)

        learn = unet_learner(data_loader, 
                            arch=getattr(models, train_arch), 
                            metrics=DiceMulti)
        learn.fine_tune(
            **fine_tune_args,
            cbs=[DVCLiveCallback(live=live)])

Although metrics, params, and plots are correctly saved for each experiment, the model is basically lost and the learn instance we are working with is just the one from the last iteration.

We should serialize the model and include it in each experiment.

This requires https://github.com/iterative/dvclive/issues/378 and is just an example of when it would be useful to have.