PaddlePaddle / PaddleNLP

👑 Easy-to-use and powerful NLP and LLM library with 🤗 Awesome model zoo, supporting wide-range of NLP tasks from research to industrial applications, including 🗂Text Classification, 🔍 Neural Search, ❓ Question Answering, ℹ️ Information Extraction, 📄 Document Intelligence, 💌 Sentiment Analysis etc.
https://paddlenlp.readthedocs.io
Apache License 2.0
12.08k stars 2.93k forks source link

paddle 文本匹配 sentence-transformers 把缓存的模型放到当前文件夹下面就不能加载了 #2872

Closed allendred closed 2 years ago

allendred commented 2 years ago

欢迎您反馈PaddleNLP使用问题,非常感谢您对PaddleNLP的贡献! 在留下您的问题时,辛苦您同步提供如下信息:

python predict.py --device gpu --params_path checkpoints/model_400/model_state.pdparams grep: warning: GREP_OPTIONS is deprecated; please use an alias or script We use pattern recognition to recognize the Tokenizer class. The 'pretrained_model_name_or_path' is /nlp_operator/sentence_transformers/bert-wwm-ext-chinese, we import <class 'paddlenlp.transformers.bert.tokenizer.BertTokenizer'>. [2022-07-26 02:45:39,947] [ INFO] - Downloading /nlp_operator/sentence_transformers/bert-wwm-ext-chinese/model_state.pdparams and saved to /nlp_operator/sentence_transformers/bert-wwm-ext-chinese Traceback (most recent call last): File "predict.py", line 156, in pretrained_model = AutoModel.from_pretrained('/nlp_operator/sentence_transformers/bert-wwm-ext-chinese')#("ernie-3.0-medium-zh") File "/usr/local/lib/python3.7/dist-packages/paddlenlp/transformers/auto/modeling.py", line 300, in from_pretrained *model_args, *kwargs) File "/usr/local/lib/python3.7/dist-packages/paddlenlp/transformers/auto/modeling.py", line 179, in _from_pretrained model_args, **kwargs) File "/usr/local/lib/python3.7/dist-packages/paddlenlp/transformers/model_utils.py", line 247, in from_pretrained file_path, default_root) File "/usr/local/lib/python3.7/dist-packages/paddlenlp/utils/downloader.py", line 155, in get_path_from_url assert is_url(url), "downloading from {} not a url".format(url) AssertionError: downloading from /nlp_operator/sentence_transformers/bert-wwm-ext-chinese/model_state.pdparams not a url

allendred commented 2 years ago

paddle2onnx 0.9.8 paddlefsl 1.1.0 paddlenlp 2.2.5 or 2.3.4 paddlepaddle-gpu 2.2.2.post112

w5688414 commented 2 years ago

使用的那个人predict.py文件。是这个吗?https://github.com/PaddlePaddle/PaddleNLP/blob/develop/examples/text_matching/sentence_transformers/predict.py

allendred commented 2 years ago

使用的那个人predict.py文件。是这个吗?https://github.com/PaddlePaddle/PaddleNLP/blob/develop/examples/text_matching/sentence_transformers/predict.py 是的呢,就是这个,我把模型名称换掉了

w5688414 commented 2 years ago

使用的那个人predict.py文件。是这个吗?https://github.com/PaddlePaddle/PaddleNLP/blob/develop/examples/text_matching/sentence_transformers/predict.py 是的呢,就是这个,我把模型名称换掉了

参考这个

# Load from local directory path
model = AutoModel.from_pretrained('./my_bert/')
print(type(model))
# <class 'paddlenlp.transformers.bert.modeling.BertModel'>

你换成下面这个试试:

pretrained_model = AutoModel.from_pretrained('/nlp_operator/sentence_transformers/bert-wwm-ext-chinese/')
allendred commented 2 years ago

/nlp_operator/sentence_transformers/bert-wwm-ext-chinese/

我找到原因了 应该把文件夹下面的模型文件名字bert-wwm-ext-chinese.pdparams 改成 model_state.pdparams,就可以了