search

moussaKam / BARThez

A french sequence to sequence pretrained model
Apache License 2.0
57 stars 11 forks source link

AttributeError when trying to load barthez or mbarthez tokenizers #3

Closed ell-hol closed 3 years ago

ell-hol commented 3 years ago

barthez_tokenizer = AutoTokenizer.from_pretrained("moussaKam/barthez") Traceback (most recent call last): File "", line 1, in File "/home/ell-hol/Desktop/envs/transformers-torch/lib/python3.8/site-packages/transformers/models/auto/tokenization_auto.py", line 370, in from_pretrained tokenizer_class = tokenizer_class_from_name(tokenizer_class_candidate) File "/home/ell-hol/Desktop/envs/transformers-torch/lib/python3.8/site-packages/transformers/models/auto/tokenization_auto.py", line 273, in tokenizer_class_from_name if c.name == class_name: AttributeError: 'NoneType' object has no attribute 'name

barthez_tokenizer = AutoTokenizer.from_pretrained("moussaKam/mbarthez")

Traceback (most recent call last): File "", line 1, in File "/home/ell-hol/Desktop/envs/transformers-torch/lib/python3.8/site-packages/transformers/models/auto/tokenization_auto.py", line 370, in from_pretrained tokenizer_class = tokenizer_class_from_name(tokenizer_class_candidate) File "/home/ell-hol/Desktop/envs/transformers-torch/lib/python3.8/site-packages/transformers/models/auto/tokenization_auto.py", line 273, in tokenizer_class_from_name if c.name == class_name: AttributeError: 'NoneType' object has no attribute 'name'

Transformers version: 4.3.2 Torch version: '1.7.1+cu101'

moussaKam commented 3 years ago

Hi @ell-hol can you try to install sentencepiece please. pip install sentencepiece

ell-hol commented 3 years ago

Installing sentencepiece did fix it. Thanks a lot !