search

fastnlp / fastNLP

fastNLP: A Modularized and Extensible NLP Framework. Currently still in incubation.
https://gitee.com/fastnlp/fastNLP
Apache License 2.0
3.06k stars 450 forks source link

encoder模块DotAttention类中softmax的bug #254

Closed keezen closed 4 years ago

keezen commented 4 years ago

https://github.com/fastnlp/fastNLP/blob/980aba9898d2c33689b88ad41f9cf173ef9e2e31/fastNLP/modules/encoder/attention.py#L29

此处应为dim=-1，否则在四维情况下就是错误的

@xuyige @choosewhatulike @yhcc

xuyige commented 4 years ago

这里一般来说作为Transformer的DotAttention大部分情况应该是三维张量。后续会修改为-1