PaddlePaddle / PaddleNLP

👑 Easy-to-use and powerful NLP and LLM library with 🤗 Awesome model zoo, supporting wide-range of NLP tasks from research to industrial applications, including 🗂Text Classification, 🔍 Neural Search, ❓ Question Answering, ℹ️ Information Extraction, 📄 Document Intelligence, 💌 Sentiment Analysis etc.
https://paddlenlp.readthedocs.io
Apache License 2.0
11.79k stars 2.88k forks source link

[Bug]: AutoModel加载本地路径模型报错 #8476

Open ZTurboX opened 2 months ago

ZTurboX commented 2 months ago

软件环境

- paddlepaddle:2.6.1
- paddlepaddle-gpu: 2.6.1
- paddlenlp: 2.7.2

重复问题

错误描述

使用AutoModel.from_pretrained(args.model_name_or_path)报错
AttributeError: Unable to parse 'architectures' or 'init_class' from ./pretrained_models/rocketqa-zh-base-para-encoder/config.json. Also unable to infer model class from 'pretrained_model_name_or_path'

稳定复现步骤 & 代码

加载模型代码:pretrained_model = AutoModel.from_pretrained(args.model_name_or_path) args.model_name_or_path为自定义本地路径,使用rocketqa-zh-base-para-encoder模型,其中config.json文件为{ "attention_probs_dropout_prob": 0.1, "fuse": false, "hidden_act": "gelu", "hidden_dropout_prob": 0.1, "hidden_size": 768, "initializer_range": 0.02, "intermediate_size": 3072, "layer_norm_eps": 1e-12, "max_position_embeddings": 2048, "model_type": "ernie", "num_attention_heads": 12, "num_hidden_layers": 12, "pad_token_id": 0, "paddlenlp_version": null, "pool_act": "tanh", "task_id": 0, "task_type_vocab_size": 3, "type_vocab_size": 4, "use_task_id": true, "vocab_size": 40000 }

w5688414 commented 2 months ago

我测了一下没什么问题:

image

请参考我的环境:

absl-py==2.1.0
accessible-pygments==0.0.4
addict==2.4.0
aiofiles==23.2.1
aiohttp==3.8.4
aiosignal==1.3.1
aistudio-sdk==0.1.7
alabaster==0.7.16
aliyun-python-sdk-core==2.15.0
aliyun-python-sdk-kms==2.16.2
altair==5.2.0
annotated-types==0.6.0
anyio==3.7.1
appdirs==1.4.4
astor==0.8.1
asttokens==2.4.1
async-timeout==4.0.3
asyncio-atexit==1.0.1
attrdict==2.0.1
attrs==23.2.0
audioread==3.0.1
Babel==2.14.0
bce-python-sdk==0.9.4
beautifulsoup4==4.12.3
black==23.3.0
blinker==1.7.0
cachetools==5.3.3
cairocffi==1.6.1
CairoSVG==2.7.1
certifi==2024.2.2
cffi==1.16.0
cfgv==3.4.0
charset-normalizer==3.3.2
click==8.1.7
cloudpickle==3.0.0
colorama==0.4.6
colorlog==6.8.2
commonmark==0.9.1
contourpy==1.2.0
coverage==7.4.4
crcmod==1.7
cryptography==42.0.5
cssselect==1.2.0
cssselect2==0.7.0
cssutils==2.10.2
cupy-cuda116==10.6.0
cycler==0.12.1
Cython==3.0.10
data==0.4
dataclasses-json==0.6.4
datasets==2.18.0
decorator==5.1.1
defusedxml==0.7.1
deploy==1.9.1
dill==0.3.4
distlib==0.3.8
docker-pycreds==0.4.0
docutils==0.20.1
einops==0.7.0
emoji==2.11.0
erniebot==0.5.2
erniebot_agent==0.5.0
et-xmlfile==1.1.0
exceptiongroup==1.2.0
execnet==2.0.2
executing==2.0.1
fast-tokenizer-python==1.0.2
fastapi==0.110.0
fastrlock==0.8.2
ffmpy==0.3.2
filelock==3.13.1
fire==0.6.0
flake8==5.0.4
Flask==3.0.2
flask-babel==4.0.0
fonttools==4.49.0
frozenlist==1.4.1
fsspec==2024.2.0
ftfy==6.2.0
funcsigs==1.0.2
future==1.0.0
gast==0.5.4
gitdb==4.0.11
GitPython==3.1.43
gradio==4.19.2
gradio_client==0.10.1
greenlet==3.0.3
grpcio==1.51.3
h11==0.12.0
h5py==3.10.0
html5lib==1.1
httpcore==0.15.0
httpx==0.25.1
huggingface-hub==0.21.1
hyperopt==0.2.7
identify==2.5.35
idna==3.6
imageio==2.34.0
imagesize==1.4.1
imgaug==0.4.0
importlib_metadata==7.1.0
importlib_resources==6.1.2
iniconfig==2.0.0
ipython==8.22.1
isort==5.11.5
itsdangerous==2.1.2
jedi==0.19.1
jieba==0.42.1
Jinja2==3.1.3
jmespath==0.10.0
joblib==1.3.2
jsonlines==4.0.0
jsonpatch==1.33
jsonpointer==2.4
jsonschema==4.21.1
jsonschema-path==0.3.2
jsonschema-specifications==2023.12.1
kiwisolver==1.4.5
langchain==0.1.9
langchain-community==0.0.24
langchain-core==0.1.27
langsmith==0.1.10
lazy-object-proxy==1.10.0
lazy_loader==0.3
librosa==0.10.1
llvmlite==0.42.0
lmdb==1.4.1
loguru==0.5.3
lxml==5.2.0
Markdown==3.5.2
markdown-it-py==3.0.0
MarkupSafe==2.1.5
marshmallow==3.21.0
matplotlib==3.8.3
matplotlib-inline==0.1.6
mccabe==0.7.0
mdurl==0.1.2
modelscope==1.13.3
mpmath==1.3.0
msgpack==1.0.8
multidict==6.0.5
multiprocess==0.70.12.2
mypy==1.6.1
mypy-extensions==1.0.0
networkx==3.2.1
nodeenv==1.8.0
numba==0.59.1
numpy==1.23.5
onnx==1.16.0
openapi-schema-validator==0.6.2
openapi-spec-validator==0.7.1
opencv-contrib-python==4.6.0.66
opencv-python==4.6.0.66
opencv-python-headless==4.9.0.80
openpyxl==3.1.2
opt-einsum==3.3.0
orjson==3.9.15
oss2==2.18.4
packaging==23.2
paddle2onnx==1.1.0
paddlefsl==1.1.0
paddlenlp==2.6.1
paddleocr==2.6.1.3
paddlepaddle==2.6.1
pandas==2.2.1
parameterized==0.9.0
parso==0.8.3
pathable==0.4.3
pathspec==0.12.1
pdf2docx==0.5.8
pexpect==4.9.0
pillow==10.2.0
platformdirs==4.2.0
pluggy==1.4.0
pooch==1.8.1
pre-commit==3.7.0
premailer==3.10.0
prompt-toolkit==3.0.43
protobuf==3.20.2
psutil==5.9.8
ptyprocess==0.7.0
pure-eval==0.2.2
py4j==0.10.9.7
pyarrow==15.0.2
pyarrow-hotfix==0.6
pybind11==2.12.0
pyclipper==1.3.0.post5
pycodestyle==2.9.1
pycparser==2.21
pycryptodome==3.20.0
pydantic==1.10.11
pydantic_core==2.16.3
pydata-sphinx-theme==0.15.2
pydub==0.25.1
pyflakes==2.5.0
Pygments==2.17.2
PyMuPDF==1.20.2
pyparsing==3.1.1
pyphen==0.14.0
pytest==8.1.1
pytest-cov==5.0.0
pytest-timeout==2.3.1
pytest-xdist==3.5.0
python-dateutil==2.8.2
python-docx==1.1.0
python-multipart==0.0.9
pytz==2024.1
PyYAML==6.0.1
rapidfuzz==3.7.0
rarfile==4.1
ray==2.5.1
readthedocs-sphinx-search==0.3.2
recommonmark==0.7.1
referencing==0.31.1
regex==2024.4.16
requests==2.31.0
rfc3339-validator==0.1.4
rich==13.7.0
rouge==1.0.1
rpds-py==0.18.0
ruff==0.2.2
sacremoses==0.1.1
safetensors==0.4.2
scikit-image==0.22.0
scikit-learn==1.4.1.post1
scipy==1.9.1
semantic-version==2.10.0
sentencepiece==0.2.0
sentry-sdk==1.44.0
seqeval==1.2.2
setproctitle==1.3.3
shapely==2.0.3
shellingham==1.5.4
simplejson==3.19.2
six==1.16.0
smmap==5.0.1
sniffio==1.3.1
snowballstemmer==2.2.0
sortedcontainers==2.4.0
soundfile==0.12.1
soupsieve==2.5
soxr==0.3.7
Sphinx==7.2.6
sphinx-book-theme==1.1.2
sphinx-copybutton==0.5.2
sphinx-markdown-tables==0.0.17
sphinx-rtd-theme==2.0.0
sphinxcontrib-applehelp==1.0.8
sphinxcontrib-devhelp==1.0.6
sphinxcontrib-htmlhelp==2.0.5
sphinxcontrib-jquery==4.1
sphinxcontrib-jsmath==1.0.1
sphinxcontrib-qthelp==1.0.7
sphinxcontrib-serializinghtml==1.1.10
SQLAlchemy==2.0.27
stack-data==0.6.3
starlette==0.36.3
sympy==1.12
tenacity==8.2.3
tensorboard==2.16.2
tensorboard-data-server==0.7.2
tensorboardX==2.6.2.2
termcolor==2.4.0
threadpoolctl==3.3.0
tifffile==2024.2.12
tiktoken==0.6.0
tinycss2==1.2.1
tokenize-rt==5.2.0
tokenizers==0.15.2
tomli==2.0.1
tomlkit==0.12.0
tool-helpers==0.1.1
toolz==0.12.1
torch==2.2.2
tqdm==4.66.2
traitlets==5.14.1
transformers==4.39.3
triton==2.2.0
typer==0.9.0
types-beautifulsoup4==4.12.0.20240106
types-html5lib==1.1.11.20240228
types-PyYAML==6.0.12.12
types-requests==2.31.0.2
types-urllib3==1.26.25.14
typing-inspect==0.9.0
typing_extensions==4.10.0
tzdata==2024.1
Unidecode==1.3.8
urllib3==1.26.2
uvicorn==0.27.1
virtualenv==20.25.1
visualdl==2.5.3
wandb==0.16.5
wcwidth==0.2.13
WeasyPrint==52.5
webencodings==0.5.1
websockets==11.0.3
Werkzeug==3.0.1
wget==3.2
xxhash==3.4.1
yacs==0.1.8
yapf==0.40.2
yarl==1.9.4
zhon==2.0.2
zipp==3.18.1
ZTurboX commented 2 months ago

我测了一下没什么问题:

image

请参考我的环境:

absl-py==2.1.0
accessible-pygments==0.0.4
addict==2.4.0
aiofiles==23.2.1
aiohttp==3.8.4
aiosignal==1.3.1
aistudio-sdk==0.1.7
alabaster==0.7.16
aliyun-python-sdk-core==2.15.0
aliyun-python-sdk-kms==2.16.2
altair==5.2.0
annotated-types==0.6.0
anyio==3.7.1
appdirs==1.4.4
astor==0.8.1
asttokens==2.4.1
async-timeout==4.0.3
asyncio-atexit==1.0.1
attrdict==2.0.1
attrs==23.2.0
audioread==3.0.1
Babel==2.14.0
bce-python-sdk==0.9.4
beautifulsoup4==4.12.3
black==23.3.0
blinker==1.7.0
cachetools==5.3.3
cairocffi==1.6.1
CairoSVG==2.7.1
certifi==2024.2.2
cffi==1.16.0
cfgv==3.4.0
charset-normalizer==3.3.2
click==8.1.7
cloudpickle==3.0.0
colorama==0.4.6
colorlog==6.8.2
commonmark==0.9.1
contourpy==1.2.0
coverage==7.4.4
crcmod==1.7
cryptography==42.0.5
cssselect==1.2.0
cssselect2==0.7.0
cssutils==2.10.2
cupy-cuda116==10.6.0
cycler==0.12.1
Cython==3.0.10
data==0.4
dataclasses-json==0.6.4
datasets==2.18.0
decorator==5.1.1
defusedxml==0.7.1
deploy==1.9.1
dill==0.3.4
distlib==0.3.8
docker-pycreds==0.4.0
docutils==0.20.1
einops==0.7.0
emoji==2.11.0
erniebot==0.5.2
erniebot_agent==0.5.0
et-xmlfile==1.1.0
exceptiongroup==1.2.0
execnet==2.0.2
executing==2.0.1
fast-tokenizer-python==1.0.2
fastapi==0.110.0
fastrlock==0.8.2
ffmpy==0.3.2
filelock==3.13.1
fire==0.6.0
flake8==5.0.4
Flask==3.0.2
flask-babel==4.0.0
fonttools==4.49.0
frozenlist==1.4.1
fsspec==2024.2.0
ftfy==6.2.0
funcsigs==1.0.2
future==1.0.0
gast==0.5.4
gitdb==4.0.11
GitPython==3.1.43
gradio==4.19.2
gradio_client==0.10.1
greenlet==3.0.3
grpcio==1.51.3
h11==0.12.0
h5py==3.10.0
html5lib==1.1
httpcore==0.15.0
httpx==0.25.1
huggingface-hub==0.21.1
hyperopt==0.2.7
identify==2.5.35
idna==3.6
imageio==2.34.0
imagesize==1.4.1
imgaug==0.4.0
importlib_metadata==7.1.0
importlib_resources==6.1.2
iniconfig==2.0.0
ipython==8.22.1
isort==5.11.5
itsdangerous==2.1.2
jedi==0.19.1
jieba==0.42.1
Jinja2==3.1.3
jmespath==0.10.0
joblib==1.3.2
jsonlines==4.0.0
jsonpatch==1.33
jsonpointer==2.4
jsonschema==4.21.1
jsonschema-path==0.3.2
jsonschema-specifications==2023.12.1
kiwisolver==1.4.5
langchain==0.1.9
langchain-community==0.0.24
langchain-core==0.1.27
langsmith==0.1.10
lazy-object-proxy==1.10.0
lazy_loader==0.3
librosa==0.10.1
llvmlite==0.42.0
lmdb==1.4.1
loguru==0.5.3
lxml==5.2.0
Markdown==3.5.2
markdown-it-py==3.0.0
MarkupSafe==2.1.5
marshmallow==3.21.0
matplotlib==3.8.3
matplotlib-inline==0.1.6
mccabe==0.7.0
mdurl==0.1.2
modelscope==1.13.3
mpmath==1.3.0
msgpack==1.0.8
multidict==6.0.5
multiprocess==0.70.12.2
mypy==1.6.1
mypy-extensions==1.0.0
networkx==3.2.1
nodeenv==1.8.0
numba==0.59.1
numpy==1.23.5
onnx==1.16.0
openapi-schema-validator==0.6.2
openapi-spec-validator==0.7.1
opencv-contrib-python==4.6.0.66
opencv-python==4.6.0.66
opencv-python-headless==4.9.0.80
openpyxl==3.1.2
opt-einsum==3.3.0
orjson==3.9.15
oss2==2.18.4
packaging==23.2
paddle2onnx==1.1.0
paddlefsl==1.1.0
paddlenlp==2.6.1
paddleocr==2.6.1.3
paddlepaddle==2.6.1
pandas==2.2.1
parameterized==0.9.0
parso==0.8.3
pathable==0.4.3
pathspec==0.12.1
pdf2docx==0.5.8
pexpect==4.9.0
pillow==10.2.0
platformdirs==4.2.0
pluggy==1.4.0
pooch==1.8.1
pre-commit==3.7.0
premailer==3.10.0
prompt-toolkit==3.0.43
protobuf==3.20.2
psutil==5.9.8
ptyprocess==0.7.0
pure-eval==0.2.2
py4j==0.10.9.7
pyarrow==15.0.2
pyarrow-hotfix==0.6
pybind11==2.12.0
pyclipper==1.3.0.post5
pycodestyle==2.9.1
pycparser==2.21
pycryptodome==3.20.0
pydantic==1.10.11
pydantic_core==2.16.3
pydata-sphinx-theme==0.15.2
pydub==0.25.1
pyflakes==2.5.0
Pygments==2.17.2
PyMuPDF==1.20.2
pyparsing==3.1.1
pyphen==0.14.0
pytest==8.1.1
pytest-cov==5.0.0
pytest-timeout==2.3.1
pytest-xdist==3.5.0
python-dateutil==2.8.2
python-docx==1.1.0
python-multipart==0.0.9
pytz==2024.1
PyYAML==6.0.1
rapidfuzz==3.7.0
rarfile==4.1
ray==2.5.1
readthedocs-sphinx-search==0.3.2
recommonmark==0.7.1
referencing==0.31.1
regex==2024.4.16
requests==2.31.0
rfc3339-validator==0.1.4
rich==13.7.0
rouge==1.0.1
rpds-py==0.18.0
ruff==0.2.2
sacremoses==0.1.1
safetensors==0.4.2
scikit-image==0.22.0
scikit-learn==1.4.1.post1
scipy==1.9.1
semantic-version==2.10.0
sentencepiece==0.2.0
sentry-sdk==1.44.0
seqeval==1.2.2
setproctitle==1.3.3
shapely==2.0.3
shellingham==1.5.4
simplejson==3.19.2
six==1.16.0
smmap==5.0.1
sniffio==1.3.1
snowballstemmer==2.2.0
sortedcontainers==2.4.0
soundfile==0.12.1
soupsieve==2.5
soxr==0.3.7
Sphinx==7.2.6
sphinx-book-theme==1.1.2
sphinx-copybutton==0.5.2
sphinx-markdown-tables==0.0.17
sphinx-rtd-theme==2.0.0
sphinxcontrib-applehelp==1.0.8
sphinxcontrib-devhelp==1.0.6
sphinxcontrib-htmlhelp==2.0.5
sphinxcontrib-jquery==4.1
sphinxcontrib-jsmath==1.0.1
sphinxcontrib-qthelp==1.0.7
sphinxcontrib-serializinghtml==1.1.10
SQLAlchemy==2.0.27
stack-data==0.6.3
starlette==0.36.3
sympy==1.12
tenacity==8.2.3
tensorboard==2.16.2
tensorboard-data-server==0.7.2
tensorboardX==2.6.2.2
termcolor==2.4.0
threadpoolctl==3.3.0
tifffile==2024.2.12
tiktoken==0.6.0
tinycss2==1.2.1
tokenize-rt==5.2.0
tokenizers==0.15.2
tomli==2.0.1
tomlkit==0.12.0
tool-helpers==0.1.1
toolz==0.12.1
torch==2.2.2
tqdm==4.66.2
traitlets==5.14.1
transformers==4.39.3
triton==2.2.0
typer==0.9.0
types-beautifulsoup4==4.12.0.20240106
types-html5lib==1.1.11.20240228
types-PyYAML==6.0.12.12
types-requests==2.31.0.2
types-urllib3==1.26.25.14
typing-inspect==0.9.0
typing_extensions==4.10.0
tzdata==2024.1
Unidecode==1.3.8
urllib3==1.26.2
uvicorn==0.27.1
virtualenv==20.25.1
visualdl==2.5.3
wandb==0.16.5
wcwidth==0.2.13
WeasyPrint==52.5
webencodings==0.5.1
websockets==11.0.3
Werkzeug==3.0.1
wget==3.2
xxhash==3.4.1
yacs==0.1.8
yapf==0.40.2
yarl==1.9.4
zhon==2.0.2
zipp==3.18.1

模型路径是填的自定义的本地路径,并不是直接填rocketqa-zh-base-para-encoder

github-actions[bot] commented 2 days ago

This issue is stale because it has been open for 60 days with no activity. 当前issue 60天内无活动,被标记为stale。