[SCRIPT] make electra pretraining runnable again

[szha]

Description

make electra pretraining runnable again

Checklist

Essentials

• [x] PR's title starts with a category (e.g. [BUGFIX], [MODEL], [TUTORIAL], [FEATURE], [DOC], etc)
• [x] Changes are complete (i.e. I finished coding on this PR)
• [x] All changes have test coverage
• [x] Code is well-documented

Changes

• [x] make electra pretraining runnable again

cc @dmlc/gluon-nlp-team

[codecov[bot]]

Codecov Report

Merging #1491 (3039f97) into master (c582b64) will increase coverage by 3.80%. The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #1491      +/-   ##
==========================================
+ Coverage   81.98%   85.78%   +3.80%     
==========================================
  Files          52       52              
  Lines        6909     6909              
==========================================
+ Hits         5664     5927     +263     
+ Misses       1245      982     -263     

Impacted Files	Coverage Δ
src/gluonnlp/data/loading.py	81.13% <0.00%> (-2.27%)	:arrow_down:
src/gluonnlp/models/mobilebert.py	87.82% <0.00%> (+0.25%)	:arrow_up:
src/gluonnlp/utils/misc.py	53.92% <0.00%> (+0.35%)	:arrow_up:
conftest.py	86.25% <0.00%> (+1.25%)	:arrow_up:
src/gluonnlp/data/tokenizers/sentencepiece.py	78.44% <0.00%> (+2.39%)	:arrow_up:
src/gluonnlp/utils/parameter.py	92.80% <0.00%> (+3.20%)	:arrow_up:
src/gluonnlp/embedding/embed_loader.py	81.52% <0.00%> (+5.73%)	:arrow_up:
src/gluonnlp/models/bert.py	94.80% <0.00%> (+7.26%)	:arrow_up:
src/gluonnlp/models/gpt2.py	98.26% <0.00%> (+7.26%)	:arrow_up:
src/gluonnlp/models/albert.py	95.43% <0.00%> (+7.71%)	:arrow_up:
... and 7 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update c582b64...3039f97. Read the comment docs.

[github-actions[bot]]

The documentation website for preview: http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR1491/fix_electra/index.html