Closed ruslangrimov closed 5 years ago
https://github.com/leaderj1001/Stand-Alone-Self-Attention/blob/98832a4a455de29e56cc19515d42275d633d5f91/attention.py#L35
You create these parameters every time you pass through forward so theirs previous values are lost. I think these parameters should be created once in __init__.
__init__
Thanks for your comment ! I will fix it as soon as possible :) Thank you so much !
https://github.com/leaderj1001/Stand-Alone-Self-Attention/blob/98832a4a455de29e56cc19515d42275d633d5f91/attention.py#L35
You create these parameters every time you pass through forward so theirs previous values are lost. I think these parameters should be created once in
__init__
.