search

dmlc / gluon-nlp

NLP made easy
https://nlp.gluon.ai/
Apache License 2.0
2.55k stars 538 forks source link

adjust implementation of relative position encoding #1470

Closed yongyi-wu closed 3 years ago

yongyi-wu commented 3 years ago

Description

This version aligns the implementation of relative position encoding with that in mesh_tensorflow. It would also help program T5 model in the future.

Changes

cc @dmlc/gluon-nlp-team

codecov[bot] commented 3 years ago

Codecov Report

Merging #1470 (d308a1d) into master (def0d70) will increase coverage by 0.00%. The diff coverage is 100.00%.

Impacted file tree graph

@@           Coverage Diff           @@
##           master    #1470   +/-   ##
=======================================
  Coverage   85.86%   85.86%           
=======================================
  Files          52       52           
  Lines        6911     6912    +1     
=======================================
+ Hits         5934     5935    +1     
  Misses        977      977
Impacted Files Coverage Δ
src/gluonnlp/op.py 95.16% <100.00%> (+0.07%) :arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update def0d70...d308a1d. Read the comment docs.

github-actions[bot] commented 3 years ago

The documentation website for preview: http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR1470/rel-embd/index.html

sxjscience commented 3 years ago

We can try to improve the docstring later.