Provide relatively small model for ViT

hpcaitech / ColossalAI-Examples

Examples of training models with hybrid parallelism using ColossalAI

Apache License 2.0

334 stars 102 forks source link

Provide relatively small model for ViT #98

Open Gy-Lu opened 2 years ago

Gy-Lu commented 2 years ago

Describe the feature

Hi, I find that we have provided too many huge models as examples, for instance, we reshape cifar-10 to 224*224 and use ViT Huge(at least not tiny) to train. This reshape is quite redundant and so as ViT Huge for cifar-10.