About mask_token in SwinTransformerForSimMIM

microsoft / Swin-Transformer

This is an official implementation for "Swin Transformer: Hierarchical Vision Transformer using Shifted Windows".

https://arxiv.org/abs/2103.14030

MIT License

13.98k stars 2.06k forks source link

About mask_token in SwinTransformerForSimMIM #347

Open YangGangZhiQi opened 9 months ago

YangGangZhiQi commented 9 months ago

Hi, thanks for this great work. After reading codes, I have a question about self.mask_token in SimMIM projects. What the role of self.mask_token in the init func of SwinTransformerForSimMIM? It sames like adding another learnable params in the model. what would happen if deleting this param? SimMIM_problems

Hope for your answer.