[TUTORIAL] Using T5 for MLM Tasks

[yongyi-wu]

Description

This PR introduces a tutorial of using pretrained T5Inference model for Masked Language Modeling (MLM) tasks. In order to smoothly handle "out-of-range" tokens, in this case <extra_id>s, we now subclass SentencepieceTokenizer to create a T5Tokenizer and adjust its decoding process. Finally, this PR renames NMTInference models (including T5 and transformer) simply into Inference models per #1501 request.

Checklist

Essentials

• [x] PR's title starts with a category (e.g. [BUGFIX], [MODEL], [TUTORIAL], [FEATURE], [DOC], etc)
• [x] Changes are complete (i.e. I finished coding on this PR)
• [x] All changes have test coverage

Changes

• [x] MLM tutorial using pretrained T5
• [x] mask_to_sentinel(), useful for MLM tasks
• [x] T5Tokenizer class
• [x] Renamed inference models
• [x] Typo fixes

cc @dmlc/gluon-nlp-team

[codecov[bot]]

Codecov Report

Merging #1504 (5d3e194) into master (4624e6b) will decrease coverage by 0.22%. The diff coverage is 63.33%.

@@            Coverage Diff             @@
##           master    #1504      +/-   ##
==========================================
- Coverage   86.59%   86.36%   -0.23%     
==========================================
  Files          54       54              
  Lines        7349     7388      +39     
==========================================
+ Hits         6364     6381      +17     
- Misses        985     1007      +22     

Impacted Files	Coverage Δ
src/gluonnlp/sequence_sampler.py	86.77% <ø> (ø)
src/gluonnlp/models/transformer.py	98.52% <50.00%> (-0.42%)	:arrow_down:
src/gluonnlp/models/t5.py	93.65% <64.81%> (-4.48%)	:arrow_down:
src/gluonnlp/data/tokenizers/yttm.py	81.89% <0.00%> (-0.87%)	:arrow_down:

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 4624e6b...5d3e194. Read the comment docs.

[github-actions[bot]]

The documentation website for preview: http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR1504/88e4c66cebce57950861258690b76f4899344de3/index.html

[github-actions[bot]]

The documentation website for preview: http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR1504/5d3e1943e57a50d72b92dc355f0ccc5db3f15531/index.html