question about detail--drop_pro parameter sd

xxxnell / how-do-vits-work

(ICLR 2022 Spotlight) Official PyTorch implementation of "How Do Vision Transformers Work?"

https://arxiv.org/abs/2202.06709

Apache License 2.0

798 stars 77 forks source link

question about detail--drop_pro parameter sd #24

Closed salt-fisher closed 1 year ago

salt-fisher commented 1 year ago

thanks for your great work! notice that you have set drop some value during the training with sd=0.1。 did you do some exps to analysis the influence between the drop ratio ?

xxxnell commented 1 year ago

Hi, @salt-fisher Thank you for your kind words!

No, I didn't analyze the drop ratio. But I think it would be interesting to conduct experiments on it.