PaddlePaddle / PaddleNLP

👑 Easy-to-use and powerful NLP and LLM library with 🤗 Awesome model zoo, supporting wide-range of NLP tasks from research to industrial applications, including 🗂Text Classification, 🔍 Neural Search, ❓ Question Answering, ℹ️ Information Extraction, 📄 Document Intelligence, 💌 Sentiment Analysis etc.
https://paddlenlp.readthedocs.io
Apache License 2.0
12.19k stars 2.95k forks source link

[Bug]: NER分析GPU环境使用CPU报错,提示(InvalidArgument) Variable value (input) of OP(fluid.layers.embedding) #8326

Closed done520 closed 6 months ago

done520 commented 7 months ago

软件环境

Linux 系统:
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 530.30.02              Driver Version: 530.30.02    CUDA Version: 12.1     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                  Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf            Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA GeForce RTX 2080 Ti      On | 00000000:02:00.0 Off |                  N/A |
| 40%   32C    P8               10W / 260W|      3MiB / 11264MiB |      0%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+

$ pip list | grep "paddle"
paddle2onnx                        1.0.1
paddlefsl                          1.1.0
paddlenlp                          2.5.2
paddlepaddle                       2.5.2
paddlepaddle-gpu                   2.5.2
WARNING: There was an error checking the latest version of pip.

重复问题

错误描述

File "./seg.py", line 214, in get_keytags
    tags = get_wordTags(doc)
  File "./seg.py", line 172, in get_wordTags
    tags = NER(doc)
  File "/home/user/anaconda3/lib/python3.7/site-packages/paddlenlp/taskflow/taskflow.py", line 850, in __call__
    results = self.task_instance(inputs)
  File "/home/user/anaconda3/lib/python3.7/site-packages/paddlenlp/taskflow/task.py", line 516, in __call__
    outputs = self._run_model(inputs)
  File "/home/user/anaconda3/lib/python3.7/site-packages/paddlenlp/taskflow/knowledge_mining.py", line 464, in _run_model
    self.predictor.run()
ValueError: (InvalidArgument) Variable value (input) of OP(fluid.layers.embedding) expected >= 0 and < 512, but got 4440227630765077628. Please check input value.
  [Hint: Expected ids[i] < row_number, but received ids[i]:4440227630765077628 >= row_number:512.] (at ../paddle/phi/kernels/cpu/embedding_kernel.cc:67)
  [operator < lookup_table_v2 > error]

稳定复现步骤 & 代码

import paddle from paddlenlp import Taskflow paddle.set_device('cpu') NER = Taskflow('ner', user_dict=user_dict, entity_only=True, device_id=-1) doc = "测试分词" tags = NER(doc) print(tags)

w5688414 commented 7 months ago

我测了一下没啥问题:

>>> tags = NER("测试分词")
>>> tags
[('测试分词', '事件类')]

以下是我的环境:

image

可以参考我的配置:

absl-py                        2.1.0
aiofiles                       23.2.1
aiohttp                        3.9.3
aiosignal                      1.3.1
aistudio-sdk                   0.1.7
altair                         5.2.0
annotated-types                0.6.0
antlr4-python3-runtime         4.9.3
anyio                          3.7.1
argon2-cffi                    23.1.0
argon2-cffi-bindings           21.2.0
astor                          0.8.1
asttokens                      2.4.1
async-timeout                  4.0.3
attrdict                       2.0.1
attrs                          23.2.0
audioread                      3.0.1
azure-core                     1.29.1
azure-storage-blob             12.19.1
Babel                          2.14.0
backcall                       0.2.0
bce-python-sdk                 0.9.4
beautifulsoup4                 4.12.3
blinker                        1.7.0
blis                           0.7.11
boilerpy3                      1.0.7
bokeh                          3.1.1
boltons                        23.1.1
Bottleneck                     1.3.8
braceexpand                    0.1.7
Brotli                         1.1.0
cachetools                     5.3.3
catalogue                      2.0.10
certifi                        2024.2.2
cffi                           1.16.0
charset-normalizer             3.3.2
click                          8.0.0
cloudpathlib                   0.16.0
colorama                       0.4.6
coloredlogs                    15.0.1
colorlog                       6.8.2
confection                     0.1.4
contourpy                      1.1.1
cryptography                   42.0.5
cssselect                      1.2.0
cssutils                       2.10.2
cuda-python                    12.3.0
cycler                         0.12.1
cymem                          2.0.8
Cython                         3.0.8
datasets                       2.17.1
decorator                      5.1.1
dill                           0.3.4
Distance                       0.1.3
easydict                       1.12
editdistance                   0.8.1
einops                         0.7.0
elasticsearch                  7.11.0
environs                       9.5.0
et-xmlfile                     1.1.0
Events                         0.5
exceptiongroup                 1.2.0
executing                      2.0.1
faiss-cpu                      1.7.4
fastapi                        0.103.2
ffmpy                          0.3.2
filelock                       3.13.1
fire                           0.6.0
Flask                          2.2.5
Flask-Babel                    2.0.0
flatbuffers                    23.5.26
fonttools                      4.49.0
frozenlist                     1.4.1
fsspec                         2023.10.0
ftfy                           6.1.3
future                         1.0.0
g2p-en                         2.1.0
g2pM                           0.1.2.5
gevent                         24.2.1
geventhttpclient               2.0.2
gradio                         4.19.2
gradio_client                  0.10.1
greenlet                       3.0.3
grpcio                         1.60.0
gunicorn                       21.2.0
h11                            0.14.0
h5py                           3.10.0
httpcore                       1.0.4
httpx                          0.27.0
huggingface-hub                0.21.1
humanfriendly                  10.0
HyperPyYAML                    1.2.2
idna                           3.6
imageio                        2.34.0
imgaug                         0.4.0
importlib-metadata             7.0.1
importlib_resources            6.1.2
inflect                        7.0.0
intervaltree                   3.1.0
ipython                        8.12.3
isodate                        0.6.1
itsdangerous                   2.1.2
jedi                           0.19.1
jieba                          0.42.1
Jinja2                         3.1.3
joblib                         1.3.2
jsonlines                      4.0.0
jsonschema                     4.21.1
jsonschema-specifications      2023.12.1
kaldiio                        2.18.0
kiwisolver                     1.4.5
langcodes                      3.3.0
langdetect                     1.0.9
lazy_loader                    0.4
librosa                        0.9.2
llvmlite                       0.41.1
lmdb                           1.4.1
loguru                         0.7.2
lxml                           5.1.0
Markdown                       3.6
markdown-it-py                 3.0.0
MarkupSafe                     2.1.5
marshmallow                    3.21.1
matplotlib                     3.7.5
matplotlib-inline              0.1.6
mdurl                          0.1.2
mido                           1.3.2
minio                          7.2.5
mmh3                           4.1.0
mock                           5.1.0
more-itertools                 10.2.0
mpmath                         1.3.0
multidict                      6.0.5
multiprocess                   0.70.12.2
murmurhash                     1.0.10
nara-wpe                       0.0.9
networkx                       3.1
nltk                           3.8.1
note-seq                       0.0.5
numba                          0.58.1
numpy                          1.22.0
omegaconf                      2.3.0
onnx                           1.15.0
onnxruntime                    1.17.1
OpenCC                         1.1.7
opencc-python-reimplemented    0.1.7
opencv-contrib-python          4.6.0.66
opencv-contrib-python-headless 4.9.0.80
opencv-python                  4.6.0.66
opencv-python-headless         4.9.0.80
openpyxl                       3.1.2
opt-einsum                     3.3.0
orjson                         3.9.15
packaging                      23.2
paddle-bfloat                  0.1.7
paddle-pipelines               0.6.2
paddle2onnx                    1.1.0
paddleaudio                    1.1.0
paddlefsl                      1.1.0
paddlehub                      2.4.0
paddlenlp                      2.5.2
paddleocr                      2.6.1.3
paddlepaddle-gpu               2.5.2.post117
paddlesde                      0.2.5
paddleslim                     2.6.0
paddlespeech                   1.4.1
paddlespeech-ctcdecoders       0.2.1
paddlespeech-feat              0.1.0
pandas                         2.0.3
parameterized                  0.9.0
parso                          0.8.3
pathos                         0.2.8
pattern_singleton              1.2.0
pdf2docx                       0.5.8
pdf2image                      1.17.0
pdfminer.six                   20231228
pdfplumber                     0.11.0
pexpect                        4.9.0
pickleshare                    0.7.5
pillow                         10.2.0
pip                            24.0
pkgutil_resolve_name           1.3.10
platformdirs                   4.2.0
pooch                          1.8.1
portalocker                    2.8.2
pox                            0.3.4
ppdiffusers                    0.19.4
ppft                           1.7.6.8
praatio                        5.1.1
premailer                      3.10.0
preshed                        3.0.9
pretty_midi                    0.2.10
prettytable                    3.10.0
prompt-toolkit                 3.0.43
protobuf                       4.25.3
psutil                         5.9.8
ptyprocess                     0.7.0
pure-eval                      0.2.2
pyarrow                        15.0.0
pyarrow-hotfix                 0.6
pybind11                       2.11.1
pyclipper                      1.3.0.post5
pycparser                      2.21
pycryptodome                   3.20.0
pydantic                       1.10.15
pydantic_core                  2.16.3
pydub                          0.25.1
Pygments                       2.17.2
pygtrie                        2.5.0
pymilvus                       2.4.0
PyMuPDF                        1.20.2
pyparsing                      3.1.1
pypdfium2                      4.29.0
pypinyin                       0.44.0
pypinyin-dict                  0.7.0
python-dateutil                2.8.2
python-docx                    1.1.0
python-dotenv                  1.0.1
python-multipart               0.0.9
python-rapidjson               1.16
pytz                           2024.1
PyWavelets                     1.4.1
pyworld                        0.3.4
PyYAML                         6.0.1
pyzmq                          25.1.2
rapidfuzz                      3.8.1
rarfile                        4.1
referencing                    0.33.0
regex                          2023.12.25
requests                       2.31.0
requests-mock                  1.11.0
resampy                        0.4.2
rich                           13.7.0
rpds-py                        0.18.0
ruamel.yaml                    0.18.6
ruamel.yaml.clib               0.2.8
ruff                           0.2.2
sacrebleu                      2.4.0
safetensors                    0.4.2
scikit-image                   0.21.0
scikit-learn                   1.3.2
scipy                          1.10.1
semantic-version               2.10.0
sentencepiece                  0.2.0
seqeval                        1.2.2
setuptools                     69.1.1
shapely                        2.0.4
shellingham                    1.5.4
six                            1.16.0
smart-open                     6.4.0
sniffio                        1.3.1
sortedcontainers               2.4.0
soundfile                      0.12.1
soupsieve                      2.5
spacy                          3.7.4
spacy-legacy                   3.0.12
spacy-loggers                  1.0.5
SQLAlchemy                     1.4.52
SQLAlchemy-Utils               0.41.2
srsly                          2.4.8
sseclient-py                   1.7.2
stack-data                     0.6.3
starlette                      0.27.0
swig                           4.2.1
sympy                          1.12
tabulate                       0.9.0
termcolor                      2.4.0
TextGrid                       1.5
thinc                          8.2.3
threadpoolctl                  3.3.0
tifffile                       2023.7.10
timer                          0.2.2
ToJyutping                     0.2.3
tomlkit                        0.12.0
tool-helpers                   0.1.1
toolz                          0.12.1
tornado                        6.4
tqdm                           4.66.2
traitlets                      5.14.1
trampoline                     0.1.2
tritonclient                   2.41.1
typeguard                      2.13.3
typer                          0.9.0
typing_extensions              4.5.0
tzdata                         2024.1
ujson                          5.9.0
urllib3                        1.26.18
uvicorn                        0.27.1
visualdl                       2.4.2
wasabi                         1.1.2
wcwidth                        0.2.13
weasel                         0.3.4
webrtcvad                      2.0.10
websockets                     11.0.3
Werkzeug                       3.0.1
wheel                          0.42.0
wordcloud                      1.8.2.2
xxhash                         3.4.1
xyzservices                    2023.10.1
yacs                           0.1.8
yarl                           1.9.4
zhon                           2.0.2
zipp                           3.17.0
zope.event                     5.0
zope.interface                 6.3