[BUGFIX] fix NTA implementation

[xcgoner]

Description

Resolve the issue https://github.com/dmlc/gluon-nlp/issues/1253 Currently, the average trigger checks “val_L > min(valid_losses[-n:])”. In this patch, I change the implementation into the version published in ICLR, [1], the algorithms is actually “val_L > min(valid_losses[:-n])”, which is also used in Salesforce’s source code: [2]

References

[1] https://openreview.net/pdf?id=SyyGPP0TZ [2] https://github.com/salesforce/awd-lstm-lm/blob/32fcb42562aeb5c7e6c9dec3f2a3baaaf68a5cb5/main.py#L275

cc @dmlc/gluon-nlp-team

[mli]

Job PR-1277/1 is complete. Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-1277/1/index.html

[codecov[bot]]

Codecov Report

Merging #1277 into master will increase coverage by 0.27%. The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #1277      +/-   ##
==========================================
+ Coverage   87.45%   87.72%   +0.27%     
==========================================
  Files          81       81              
  Lines        7365     7365              
==========================================
+ Hits         6441     6461      +20     
+ Misses        924      904      -20     

Impacted Files	Coverage Δ
src/gluonnlp/data/word_embedding_evaluation.py	96.93% <0.00%> (+7.66%)	:arrow_up:

[xcgoner]

@sxjscience @szha I've updated the results reported in the webpage in this pr. Although the results of language models look fine to me, the cached language models seem to have a significant performance regression. The updated logs could be found in https://github.com/dmlc/web-data/pull/259