Fix attn bias - Githubissues

dmlc / gluon-nlp

NLP made easy

https://nlp.gluon.ai/

Apache License 2.0

2.56k stars 538 forks source link

Fix attn bias #1296

Closed eric-haibin-lin closed 4 years ago

eric-haibin-lin commented 4 years ago

Description

Fixes https://github.com/dmlc/gluon-nlp/issues/1183 https://github.com/dmlc/gluon-nlp/issues/1294 @davisliang @kaonashi-tyc

Checklist

Essentials

[ ] PR's title starts with a category (e.g. [BUGFIX], [MODEL], [TUTORIAL], [FEATURE], [DOC], etc)
[ ] Changes are complete (i.e. I finished coding on this PR)
[ ] All changes have test coverage
[ ] Code is well-documented

Changes

[ ] Feature1, tests, (and when applicable, API doc)
[ ] Feature2, tests, (and when applicable, API doc)

Comments

If this change is a backward incompatible change, why must this change be made.
Interesting edge cases to note here

cc @dmlc/gluon-nlp-team

szha commented 4 years ago

@eric-haibin-lin thanks for the patch. shall we add this to v0.9.x?