The pos_embed's dimension of checkpoint `TinyMIM-FT-Tstar.pth`

[RockeyCoss]

Thanks for your wonderful work! By the way, could you please tell me why the length of the pos_embed parameters in the checkpoint TinyMIM-FT-Tstar.pth is 198(14*14+2) instead of 197(14*14+1)? Thank you very much!

[OliverRensu]

We take knowledge distillation like DeiT in finetuning TinyMIM-Tstar, therefore, there is an extra distillation token.

[RockeyCoss]

Thank you very much! Is the first token the distillation token?

[OliverRensu]

No, the second token is the distillation token. You can refer https://github.com/OliverRensu/TinyMIM/blob/master/Segmentation/TinyMIMstar-T/backbone/mae.py

[RockeyCoss]

I see, thank you very much!