Hello!
I was trying to finetune your model on google colab, but I stumbled upon a "curious" error
Traceback (most recent call last):
File "pretrain_megatron.py", line 714, in <module>
main()
File "pretrain_megatron.py", line 659, in main
args.eod_token = get_train_val_test_data(args)
File "pretrain_megatron.py", line 594, in get_train_val_test_data
args)
File "/content/ru-gpts/configure_data.py", line 34, in apply
return make_loaders(args)
File "/content/ru-gpts/configure_data.py", line 170, in make_loaders
train, tokenizer = data_utils.make_dataset(**data_set_args)
File "/content/ru-gpts/data_utils/__init__.py", line 101, in make_dataset
pad_token, character_converage, **kwargs)
File "/content/ru-gpts/data_utils/tokenization.py", line 43, in make_tokenizer
return GPT2BPETokenizer(model_path=model_path, **kwargs)
File "/content/ru-gpts/data_utils/tokenization.py", line 823, in __init__
self.text_tokenizer = GPT2Tokenizer.from_pretrained(model_path, cache_dir=cache_dir)
File "/usr/local/lib/python3.6/dist-packages/transformers/tokenization_utils.py", line 393, in from_pretrained
return cls._from_pretrained(*inputs, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/transformers/tokenization_utils.py", line 544, in _from_pretrained
tokenizer = cls(*init_inputs, **init_kwargs)
File "/usr/local/lib/python3.6/dist-packages/transformers/tokenization_gpt2.py", line 149, in __init__
super().__init__(bos_token=bos_token, eos_token=eos_token, unk_token=unk_token, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/transformers/tokenization_utils.py", line 337, in __init__
assert isinstance(value, str)
AssertionError
Hello! I was trying to finetune your model on google colab, but I stumbled upon a "curious" error
Here are my arguments:
I've got your model from archive from gdrive What's wrong with tokenizer? It's GPT3Large