What are the hyperparameters for finetuning on cifar10/100, and other small datasets

facebookresearch / deit

Official DeiT repository

Apache License 2.0

4.02k stars 553 forks source link

What are the hyperparameters for finetuning on cifar10/100, and other small datasets #175

Open Phuoc-Hoan-Le opened 2 years ago

Phuoc-Hoan-Le commented 2 years ago

Cannot find it in the paper? Is it the same as training on imagenet?

TouvronHugo commented 1 year ago

Hi @CharlesLeeeee, Thanks for your message. No the hparams are not the same (see here for more details). Best, Hugo

kevinkasa commented 1 year ago

Hi @TouvronHugo, I noticed that those hyperparameters are from 2021, which I believe is pre DeiT III. I was wondering if you use the same finetuning hyperparameters for DeiT III, or if something else is used? Thank you!