search

JieShibo / PETL-ViT

[ICCV 2023] Binary Adapters, [AAAI 2023] FacT, [Tech report] Convpass
MIT License
162 stars 7 forks source link

The number of trainable parameters #2

Closed Zhangwenyao1 closed 1 year ago

Zhangwenyao1 commented 1 year ago

I want to know why the number of parameter is 2L ×( (2h + 1)d + 9hh + 2h)? ( especially (2h+1)d and 2h.)

JieShibo commented 1 year ago

Hi Wenyao, 1×1 Conv weight: d × h × 1 × 1, bias: h 3×3 Conv weight: h × h × 3 × 3, bias: h 1×1 Conv weight: h × d × 1 × 1 , bias: d 2 Convpass modules per layer So there are 2L((2h + 1)d + 9h^2 + 2h) in total.