search

leaderj1001 / Stand-Alone-Self-Attention

Implementing Stand-Alone Self-Attention in Vision Models using Pytorch
MIT License
456 stars 83 forks source link

rel_w and rel_h parameters #2

Closed ruslangrimov closed 5 years ago

ruslangrimov commented 5 years ago

https://github.com/leaderj1001/Stand-Alone-Self-Attention/blob/98832a4a455de29e56cc19515d42275d633d5f91/attention.py#L35

You create these parameters every time you pass through forward so theirs previous values are lost. I think these parameters should be created once in __init__.

leaderj1001 commented 5 years ago

Thanks for your comment ! I will fix it as soon as possible :) Thank you so much !