Morizeyao / GPT2-Chinese

Chinese version of GPT2 training code, using BERT tokenizer.
MIT License
7.46k stars 1.7k forks source link

Huggingface 模型无法导入 #213

Closed Syarotto closed 3 years ago

Syarotto commented 3 years ago
  1. 在联网机器上,
    from transformers import BertTokenizer, GPT2LMHeadModel, TextGenerationPipeline
    tokenizer = BertTokenizer.from_pretrained("uer/gpt2-chinese-cluecorpussmall")

    报错:

    OSError: Model name 'uer/gpt2-chinese-cluecorpussmall' was not found in tokenizers model name list (bert-base-uncased, bert-large-uncased, bert-base-cased, bert-large-cased, bert-base-multilingual-uncased, bert-base-multilingual-cased, bert-base-chinese, bert-base-german-cased, bert-large-uncased-whole-word-masking, bert-large-cased-whole-word-masking, bert-large-uncased-whole-word-masking-finetuned-squad, bert-large-cased-whole-word-masking-finetuned-squad, bert-base-cased-finetuned-mrpc, bert-base-german-dbmdz-cased, bert-base-german-dbmdz-uncased, TurkuNLP/bert-base-finnish-cased-v1, TurkuNLP/bert-base-finnish-uncased-v1, wietsedv/bert-base-dutch-cased). We assumed 'uer/gpt2-chinese-cluecorpussmall' was a path, a model identifier, or url to a directory containing vocabulary files named ['vocab.txt'] but couldn't find such vocabulary files at this path or url.
  2. 在断网机器上,手动从uer/gpt2-chinese-cluecorpussmall下载后,
    path = path_to_model
    model = GPT2LMHeadModel.from_pretrained(path)

    报错:

    OSError: Error no file named ['pytorch_model.bin', 'tf_model.h5', 'model.ckpt.index'] found in directory path or `from_tf` set to False. 

    注意到从网页下载的pytorch模型格式为.zip,是不是有什么问题? 谢谢!

JHR0717 commented 3 years ago

我在联网环境拷贝了你的代码在windows环境下和linux环境下都没有问题,估计是依赖的版本问题?可以再看看。非联网情况下模型下载得先解压一下,用pytorch_model.bin那个文件