search

asyml / vision-transformer-pytorch

Pytorch version of Vision Transformer (ViT) with pretrained models. This is part of CASL (https://casl-project.github.io/) and ASYML project.
https://asyml.io/
Apache License 2.0
342 stars 58 forks source link

parameter setting to train on ImageNet from scratch #8

Open zhiyuanli1992 opened 3 years ago

zhiyuanli1992 commented 3 years ago

Thank you for your great work. Would you please post your training scripts to train the model on ImageNet. For example the learning rate, weight decay, training_step and etc? Thank you very much!

DevilYangS commented 3 years ago

+1

cjsg commented 3 years ago

Did you actually try to train your model from scratch, or did you always use the pre-trained weights?

In particular, in the README, you write

Similar results as in original implementation are achieved.

Do you mean, when training from scratch, or when importing pre-trained weights and fine-tuning?

Hhhhhhao commented 3 years ago

The cifar results are fine-tuned from the pretrained weights, not training from scratch

zhiyuanli1992 commented 3 years ago

ar results are fine-tuned from the pretrained weights, not training from scratch

Hi, can I have the training script to train on ImageNet from scratch? Thanks!