search

asyml / vision-transformer-pytorch

Pytorch version of Vision Transformer (ViT) with pretrained models. This is part of CASL (https://casl-project.github.io/) and ASYML project.
https://asyml.io/
Apache License 2.0
342 stars 58 forks source link

fine-tune imagenet21k_ViT-B_16.npz with pre_logits? #10

Open 4vicii opened 7 months ago

4vicii commented 7 months ago

hi , im really preciated by the work. but i have a question about "pre_logits". Did you set the "representation_size" as defualt None when you finetune imagenet21k_ViT-B_16.npz with imagenet 1k?

i mean the code about models.py

if representation_size is not None:
  x = nn.Dense(x, representation_size, name='pre_logits')
  x = nn.tanh(x)
else:
  x = IdentityLayer(x, name='pre_logits')