How did you initialize TA weights?

landskape-ai / triplet-attention

Official PyTorch Implementation for "Rotate to Attend: Convolutional Triplet Attention Module." [WACV 2021]

https://openaccess.thecvf.com/content/WACV2021/html/Misra_Rotate_to_Attend_Convolutional_Triplet_Attention_Module_WACV_2021_paper.html

MIT License

406 stars 49 forks source link

How did you initialize TA weights? #23

Closed osivaz61 closed 1 year ago

digantamisra98 commented 1 year ago

Random for most standard networks, for ResNet models with the same Kaiming initialization that ResNet follows. We did not explicitly optimize to find optimal initialization for the weights.