[FEATURE]Generation script improvement

[hutao965]

Description

(refer to https://github.com/dmlc/gluon-nlp/issues/1360)

Checklist

Essentials

• [x] PR's title starts with a category (e.g. [BUGFIX], [MODEL], [TUTORIAL], [FEATURE], [DOC], etc)
• [x] Changes are complete (i.e. I finished coding on this PR)
• [ ] All changes have test coverage
• [ ] Code is well-documented

Changes

• [x] Rename 'generate' (https://github.com/dmlc/gluon-nlp/tree/master/scripts/generate) --> 'generation'
• [x] Support sampling from the TN layout, which will be faster for sequence sampling.

  time python generate_unconditional_gpt2_samples.py \
  --model_name gpt2_124M \
  --gpu 0 \
  --temperature 0.7 \
  --top_k 40 \
  --nsamples 64 \
  --batch_size 16 \
  --layout NT
  (296 seconds)

  time python generate_unconditional_gpt2_samples.py \
  --model_name gpt2_124M \
  --gpu 0 \
  --temperature 0.7 \
  --top_k 40 \
  --nsamples 64 \
  --batch_size 16 \
  --layout TN
  (268 seconds)

  this example shows about 9% speed up through TN layout, and this improvement depends on the batch_size

cc @dmlc/gluon-nlp-team

[sxjscience]

Need to add test cases in https://github.com/dmlc/gluon-nlp/blob/master/tests/test_sequence_sampler.py

[codecov[bot]]

Codecov Report

Merging #1365 into master will increase coverage by 10.73%. The diff coverage is 80.00%.

@@             Coverage Diff             @@
##           master    #1365       +/-   ##
===========================================
+ Coverage   70.76%   81.49%   +10.73%     
===========================================
  Files         105       52       -53     
  Lines       12504     6905     -5599     
===========================================
- Hits         8848     5627     -3221     
+ Misses       3656     1278     -2378     

Impacted Files	Coverage Δ
src/gluonnlp/sequence_sampler.py	86.72% <80.00%> (-0.62%)	:arrow_down:
src/gluonnlp/models/xlmr.py	58.82% <0.00%> (-29.42%)	:arrow_down:
src/gluonnlp/models/__init__.py	67.74% <0.00%> (-29.04%)	:arrow_down:
src/gluonnlp/models/electra.py	57.89% <0.00%> (-18.56%)	:arrow_down:
src/gluonnlp/models/bart.py	70.62% <0.00%> (-15.63%)	:arrow_down:
src/gluonnlp/utils/config.py	61.53% <0.00%> (-15.39%)	:arrow_down:
src/gluonnlp/models/roberta.py	84.38% <0.00%> (-9.29%)	:arrow_down:
src/gluonnlp/utils/misc.py	51.08% <0.00%> (-7.74%)	:arrow_down:
src/gluonnlp/models/albert.py	87.80% <0.00%> (-7.67%)	:arrow_down:
src/gluonnlp/models/gpt2.py	90.94% <0.00%> (-7.32%)	:arrow_down:
... and 61 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 0e86648...b2f25f0. Read the comment docs.