AssertionError: masked_positions tensor is required for decoding masked language model

[xf05888]

Description

I want to import RoBERTa in GluonNLP

Error message for the last command

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.7/dist-packages/gluonnlp/model/bert.py", line 604, in __call__
    masked_positions=masked_positions)
  File "/usr/local/lib/python3.7/dist-packages/gluonnlp/model/bert.py", line 429, in __call__
    valid_length, masked_positions)
  File "/usr/local/lib/python3.7/dist-packages/mxnet/gluon/block.py", line 573, in __call__
    out = self.forward(*args)
  File "/usr/local/lib/python3.7/dist-packages/mxnet/gluon/block.py", line 957, in forward
    return self.hybrid_forward(ndarray, x, *args, **params)
  File "/usr/local/lib/python3.7/dist-packages/gluonnlp/model/bert.py", line 462, in hybrid_forward
    'masked_positions tensor is required for decoding masked language model'
AssertionError: masked_positions tensor is required for decoding masked language model

Command I ran from http://gluon-nlp.mxnet.io/model_zoo/bert/index.html

import gluonnlp as nlp; import mxnet as mx;
model, vocab = nlp.model.get_model('roberta_12_768_12', dataset_name='openwebtext_ccnews_stories_books_cased');
tokenizer = nlp.data.GPT2BPETokenizer();
text = [vocab.bos_token] + tokenizer('Hello world!') + [vocab.eos_token];
seq_encoding = model(mx.nd.array([vocab[text]]))

[xf05888]

@mxnet-label-bot add [bug]

[eric-haibin-lin]

Ah, sorry about that. I made a last minute change and missed it. Can you try

model, vocab = nlp.model.get_model('roberta_12_768_12', dataset_name='openwebtext_ccnews_stories_books_cased', use_decoder=False);

instead? The use_decoder flag loads the weight for mask language model decoder and expected mask indices. I'll update the doc also

[xf05888]

@eric-haibin-lin Thanks, the problem is solved :smile: