Open vinwinter opened 4 years ago
send full command please
@vinwinter This issue was solved somehow or not?
I am interested in GPT2 model usage, assuming there is no option to train with gpt2-large
but it is interesting how do gpt2
behave.
And, assuming GPT2 was released in february 2019 - what is the clear path to use up-to-date data in model (i mean news, wikidata, etc collected after feb 2019) or how to implement OpenGPT2 https://www.reddit.com/r/MachineLearning/comments/cu9xgi/p_opengpt2_we_replicated_gpt2_because_you_can_too/ ?
@vinwinter This issue was solved somehow or not? I am interested in GPT2 model usage, assuming there is no option to train with
gpt2-large
but it is interesting how dogpt2
behave. And, assuming GPT2 was released in february 2019 - what is the clear path to use up-to-date data in model (i mean news, wikidata, etc collected after feb 2019) or how to implement OpenGPT2 https://www.reddit.com/r/MachineLearning/comments/cu9xgi/p_opengpt2_we_replicated_gpt2_because_you_can_too/ ?
you can python train.py --model="gpt2-large"
Has anyone found a solution yet? I tried with 'gpt2-large' as well. And it just bumps into same error:
**kwargs,
File "/home/test/miniconda3/envs/qags/lib/python3.6/site-packages/transformers/tokenization_utils.py", line 729, in _decode
sub_texts.append(self.convert_tokens_to_string(current_sub_text))
File "/home/test/miniconda3/envs/qags/lib/python3.6/site-packages/transformers/models/gpt2/tokenization_gpt2.py", line 261, in convert_tokens_to_string
text = "".join(tokens)
TypeError: sequence item 0: expected str instance, NoneType found```
Hello, I had the same problem and found that in my case I was generated token ids out of bounds of a tokenizer. Solution is to remove those:
y = model.generate(...)
y[y >= tokenizer.vocab_size] = tokenizer.unk_token_id
I get an error when I change from gpt to gpt2 in the interactive.py file. However, when I change from gpt to gpt2 in the train.py file, everything is fine.
The error I get: File "/transfer-learning-conv-ai/interact.py", line 130, in run logger.info("Selected personality: %s", tokenizer.decode(chain(*personality)))
File "/pytorch_transformers/tokenization_utils.py", line 767, in decode sub_texts.append(self.convert_tokens_to_string(current_sub_text))
File "/pytorch_transformers/tokenization_gpt2.py", line 198, in convert_tokens_to_string text = ''.join(tokens)
TypeError: sequence item 0: expected str instance, NoneType found
I also tried to switch from pytorch-transformer to the new transformers library to no avail.