fastnlp / fastNLP

fastNLP: A Modularized and Extensible NLP Framework. Currently still in incubation.
https://gitee.com/fastnlp/fastNLP
Apache License 2.0
3.05k stars 451 forks source link

StaticEmbedding 中的 dropout 问题 #417

Open XieDelong opened 2 years ago

XieDelong commented 2 years ago

在 StaticEmbedding forward 方法中, words = self.drop_word(words) words = self.embedding(words) words = self.dropout(words)

drop_word 函数有 self.training 的判断,但是 dropout 没有,导致在推断的时候,同样的 query 预测结果不一致问题

yhcc commented 2 years ago

由于 self.dropout 是nn.Dropout,它本身是受到module的training状态管控的,所以如果调用了 StaticEmbedding.eval() 的话,这个 dropout 就是自动关闭的了。