microsoft / Swin-Transformer

This is an official implementation for "Swin Transformer: Hierarchical Vision Transformer using Shifted Windows".
https://arxiv.org/abs/2103.14030
MIT License
13.98k stars 2.06k forks source link

How many parameters does Swin-Transformer-V2-giant have? #341

Open wusheng816 opened 12 months ago

wusheng816 commented 12 months ago

I made the following modifications in 'configs/simmim/simmim_pretrainswin_base__img192_window6800ep.yaml', and trained a model with 2.6 billion parameters, which is 400 million less than swin-V2-giant claimed 3 billion. I wonder how many parameters does the official version of swin-V2-giant have? Thanks! EMBED_DIM: 512 DEPTHS: [ 2, 2, 42, 2 ] NUM_HEADS: [ 16, 32, 64, 128 ]