search

GeorgeCazenavette / mtt-distillation

Official code for our CVPR '22 paper "Dataset Distillation by Matching Training Trajectories"
https://georgecazenavette.github.io/mtt-distillation/
Other
395 stars 55 forks source link

About Hyper-paramters #32

Open maple-zhou opened 1 year ago

maple-zhou commented 1 year ago

Hi, sorry to bother. I really appreciate that this is a wonderful work, but I'm wondering how do you confirm the hyper-parameters of the distill.py? e.g. lr_img, lr_lr, lr_teacher, max_start_epoch, and so on. Do you use the grid search on the hyper parameters? If so, how do you implement it? Since I notice that there are a number of hyper-parameters needing to be set, I think it would take quite a few time to get a good setting. Thanks!

GeorgeCazenavette commented 1 year ago

Hi, apologies for the late reply.

For the smaller experiments, we did a decently sized grid search. However, we could not afford to do this for the larger experiments.

I just ran a bunch of experiments and tracked them with wandb.

Ideally, the method should not be as sensitive to hparams. Hopefully future work can resolve this :)