Closed jnhwkim closed 3 years ago
I believe .sum(dim=2) should be added to the below line:
.sum(dim=2)
https://github.com/leaderj1001/Stand-Alone-Self-Attention/blob/a983f0f643632b1f2b7b8b27693182f22e9e574c/attention.py#L48
for dot-product in Equation 2 in the paper.
Duplicated with #10.
I believe
.sum(dim=2)
should be added to the below line:https://github.com/leaderj1001/Stand-Alone-Self-Attention/blob/a983f0f643632b1f2b7b8b27693182f22e9e574c/attention.py#L48
for dot-product in Equation 2 in the paper.