search

asyml / texar-pytorch

Integrating the Best of TF into PyTorch, for Machine Learning, Natural Language Processing, and Text Generation. This is part of the CASL project: http://casl-project.ai/
https://asyml.io
Apache License 2.0
745 stars 117 forks source link

XLNet tokenizer bug #273

Closed atif93 closed 4 years ago

atif93 commented 4 years ago

The assertion should pass even if the pretrained_model_name is not passed.

Instead of checking if pretrained_model_name is not None, we have to check if self.pretrained_model_name is not None. This is because pretrained_model_name is an optional argument and self.pretrained_model_name is set accordingly if pretrained_model_name is None.

gpengzhi commented 4 years ago

Thanks for this fix! I think lines 99 and 100 should also be fixed.

codecov[bot] commented 4 years ago

Codecov Report

Merging #273 into master will not change coverage. The diff coverage is 0%.

Impacted file tree graph

@@          Coverage Diff           @@
##           master    #273   +/-   ##
======================================
  Coverage    83.1%   83.1%           
======================================
  Files         195     195           
  Lines       15316   15316           
======================================
  Hits        12728   12728           
  Misses       2588    2588
Impacted Files Coverage Δ
texar/torch/data/tokenizers/xlnet_tokenizer.py 85.38% <0%> (ø) :arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update d3dd3cc...84197a6. Read the comment docs.