dmlc / gluon-nlp

NLP made easy
https://nlp.gluon.ai/
Apache License 2.0
2.55k stars 538 forks source link

[FEATURE] Implement mT5 models and conversion scripts; Add T5 documentation #1506

Closed yongyi-wu closed 3 years ago

yongyi-wu commented 3 years ago

Description

This PR introduces mT5 models and conversion scripts to GluonNLP codebase. The models, tokenizer, conversion scripts, and test cases reuse most of T5's, so the correctness largely depends on the implementation of T5's components. Besides, docstrings for all T5 building blocks have been added.

Checklist

Essentials

Changes

cc @dmlc/gluon-nlp-team

codecov[bot] commented 3 years ago

Codecov Report

Merging #1506 (eddda7d) into master (f44ec96) will decrease coverage by 0.12%. The diff coverage is 95.68%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1506      +/-   ##
==========================================
- Coverage   86.35%   86.23%   -0.13%     
==========================================
  Files          54       55       +1     
  Lines        7388     7502     +114     
==========================================
+ Hits         6380     6469      +89     
- Misses       1008     1033      +25     
Impacted Files Coverage Δ
src/gluonnlp/attention_cell.py 88.39% <ø> (ø)
src/gluonnlp/models/mt5.py 95.49% <95.49%> (ø)
src/gluonnlp/models/__init__.py 100.00% <100.00%> (ø)
src/gluonnlp/models/t5.py 93.67% <100.00%> (+0.01%) :arrow_up:
src/gluonnlp/models/transformer.py 98.52% <100.00%> (+<0.01%) :arrow_up:
src/gluonnlp/data/loading.py 75.84% <0.00%> (-7.55%) :arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update f44ec96...eddda7d. Read the comment docs.

github-actions[bot] commented 3 years ago

The documentation website for preview: http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR1506/ec54d05bd8331c5ee50b0ff169818952dd79846c/index.html

github-actions[bot] commented 3 years ago

The documentation website for preview: http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR1506/eddda7db3c822ff773658c613df8ad1128b41d87/index.html