Use tanh approximation in gelu

dmlc / gluon-nlp

NLP made easy

https://nlp.gluon.ai/

Apache License 2.0

2.56k stars 538 forks source link

Closed bgawrych closed 1 year ago

bgawrych commented 1 year ago

MXNet now supports two approximation modes for GELU. This change is enabling it for GELU block in gluon-nlp

cc @dmlc/gluon-nlp-team