CUDNN_STATUS_NOT_INITIALIZED

请提供下述完整信息以便快速定位问题/Please provide the following information to quickly locate the problem

系统环境/System Environment：

Ubuntu

DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=22.04
DISTRIB_CODENAME=jammy
DISTRIB_DESCRIPTION="Ubuntu 22.04.3 LTS"

pip list

(paddle) yuhang@a1:/opt/downloads$ pip list
Package                Version
---------------------- -----------
anyio                  4.3.0
astor                  0.8.1
attrdict               2.0.1
Babel                  2.14.0
bce-python-sdk         0.9.4
beautifulsoup4         4.12.3
blinker                1.7.0
cachetools             5.3.2
certifi                2024.2.2
charset-normalizer     3.3.2
click                  8.1.7
contourpy              1.2.0
cssselect              1.2.0
cssutils               2.9.0
cycler                 0.12.1
Cython                 3.0.8
decorator              5.1.1
easydict               1.12
et-xmlfile             1.1.0
exceptiongroup         1.2.0
faiss                  1.7.4
faiss-cpu              1.7.4
faiss-gpu              1.7.2
fire                   0.5.0
Flask                  3.0.2
flask-babel            4.0.0
fonttools              4.49.0
future                 0.18.3
gast                   0.3.3
h11                    0.14.0
httpcore               1.0.3
httpx                  0.26.0
idna                   3.6
imageio                2.34.0
imgaug                 0.4.0
itsdangerous           2.1.2
Jinja2                 3.1.3
joblib                 1.3.2
kiwisolver             1.4.5
lazy_loader            0.3
lmdb                   1.4.1
lxml                   5.1.0
MarkupSafe             2.1.5
matplotlib             3.8.3
mkl-fft                1.3.8
mkl-random             1.2.4
mkl-service            2.4.0
networkx               3.2.1
numpy                  1.26.3
opencv-contrib-python  4.6.0.66
opencv-python          4.6.0.66
opencv-python-headless 4.9.0.80
openpyxl               3.1.2
opt-einsum             3.3.0
packaging              23.2
paddleclas             0.0.0
paddleocr              2.7.0.3
paddlepaddle-gpu       2.6.0
pandas                 2.2.0
pdf2docx               0.5.8
pillow                 10.2.0
pip                    23.3.1
premailer              3.10.0
prettytable            3.9.0
protobuf               4.25.3
psutil                 5.9.8
pyclipper              1.3.0.post5
pycryptodome           3.20.0
PyMuPDF                1.20.2
pyparsing              3.1.1
python-dateutil        2.8.2
python-docx            1.1.0
pytz                   2024.1
PyYAML                 6.0.1
rapidfuzz              3.6.1
rarfile                4.1
requests               2.31.0
scikit-image           0.22.0
scikit-learn           1.4.1.post1
scipy                  1.12.0
setuptools             68.2.2
shapely                2.0.3
six                    1.16.0
sniffio                1.3.0
soupsieve              2.5
termcolor              2.4.0
threadpoolctl          3.3.0
tifffile               2024.2.12
tqdm                   4.66.2
typing_extensions      4.9.0
tzdata                 2024.1
ujson                  5.9.0
urllib3                2.2.1
visualdl               2.5.3
wcwidth                0.2.13
Werkzeug               3.0.1
wheel                  0.41.2

python是3.10.13 （conda环境）

cuda环境 +-----------------------------------------------------------------------------+ | NVIDIA-SMI 520.61.05 Driver Version: 520.61.05 CUDA Version: 11.8 | |-------------------------------+----------------------+----------------------+ CUDNN是8.7 Local Installer for Ubuntu22.04 x86_64 (Deb)

版本号/Version：Paddle： PaddleOCR：问题相关组件/Related components：

paddleclas             0.0.0
paddleocr              2.7.0.3
paddlepaddle-gpu       2.6.0

paddleclas是拉源码库安装的

运行指令/Command Code：

paddleocr --image_dir=/opt/downloads/imgs/table_white.png --type=structure --image_orientation=true

完整报错/Complete Error Message：

[2024/02/20 04:01:27] ppocr DEBUG: Namespace(help='==SUPPRESS==', use_gpu=True, use_xpu=False, use_npu=False, ir_optim=True, use_tensorrt=False, min_subgraph_size=15, precision='fp32', gpu_mem=500, gpu_id=0, image_dir='/opt/downloads/imgs/table_white.png', page_num=0, det_algorithm='DB', det_model_dir='/home/yuhang/.paddleocr/whl/det/ch/ch_PP-OCRv4_det_infer', det_limit_side_len=960, det_limit_type='max', det_box_type='quad', det_db_thresh=0.3, det_db_box_thresh=0.6, det_db_unclip_ratio=1.5, max_batch_size=10, use_dilation=False, det_db_score_mode='fast', det_east_score_thresh=0.8, det_east_cover_thresh=0.1, det_east_nms_thresh=0.2, det_sast_score_thresh=0.5, det_sast_nms_thresh=0.2, det_pse_thresh=0, det_pse_box_thresh=0.85, det_pse_min_area=16, det_pse_scale=1, scales=[8, 16, 32], alpha=1.0, beta=1.0, fourier_degree=5, rec_algorithm='SVTR_LCNet', rec_model_dir='/home/yuhang/.paddleocr/whl/rec/ch/ch_PP-OCRv4_rec_infer', rec_image_inverse=True, rec_image_shape='3, 48, 320', rec_batch_num=6, max_text_length=25, rec_char_dict_path='/home/yuhang/anaconda3/envs/paddle/lib/python3.10/site-packages/paddleocr/ppocr/utils/ppocr_keys_v1.txt', use_space_char=True, vis_font_path='./doc/fonts/simfang.ttf', drop_score=0.5, e2e_algorithm='PGNet', e2e_model_dir=None, e2e_limit_side_len=768, e2e_limit_type='max', e2e_pgnet_score_thresh=0.5, e2e_char_dict_path='./ppocr/utils/ic15_dict.txt', e2e_pgnet_valid_set='totaltext', e2e_pgnet_mode='fast', use_angle_cls=False, cls_model_dir=None, cls_image_shape='3, 48, 192', label_list=['0', '180'], cls_batch_num=6, cls_thresh=0.9, enable_mkldnn=False, cpu_threads=10, use_pdserving=False, warmup=False, sr_model_dir=None, sr_image_shape='3, 32, 128', sr_batch_num=1, draw_img_save_dir='./inference_results', save_crop_res=False, crop_res_save_dir='./output', use_mp=False, total_process_num=1, process_id=0, benchmark=False, save_log_path='./log_output/', show_log=True, use_onnx=False, output='./output', table_max_len=488, table_algorithm='TableAttn', table_model_dir='/home/yuhang/.paddleocr/whl/table/ch_ppstructure_mobile_v2.0_SLANet_infer', merge_no_span_structure=True, table_char_dict_path='/home/yuhang/anaconda3/envs/paddle/lib/python3.10/site-packages/paddleocr/ppocr/utils/dict/table_structure_dict_ch.txt', layout_model_dir='/home/yuhang/.paddleocr/whl/layout/picodet_lcnet_x1_0_fgd_layout_cdla_infer', layout_dict_path='/home/yuhang/anaconda3/envs/paddle/lib/python3.10/site-packages/paddleocr/ppocr/utils/dict/layout_dict/layout_cdla_dict.txt', layout_score_threshold=0.5, layout_nms_threshold=0.5, kie_algorithm='LayoutXLM', ser_model_dir=None, re_model_dir=None, use_visual_backbone=True, ser_dict_path='../train_data/XFUND/class_list_xfun.txt', ocr_order_method=None, mode='structure', image_orientation=True, layout=True, table=True, ocr=True, recovery=False, use_pdf2docx_api=False, invert=False, binarize=False, alphacolor=(255, 255, 255), lang='ch', det=True, rec=True, type='structure', ocr_version='PP-OCRv4', structure_version='PP-StructureV2')
[2024/02/20 04:01:34] ppocr INFO: **********/opt/downloads/imgs/table_white.png**********
[2024/02/20 04:01:34] ppocr INFO: processing 1/1 page:
Traceback (most recent call last):
File "/home/yuhang/anaconda3/envs/paddle/bin/paddleocr", line 8, in <module>
sys.exit(main())
File "/home/yuhang/anaconda3/envs/paddle/lib/python3.10/site-packages/paddleocr/paddleocr.py", line 837, in main
result = engine(img, img_idx=index)
File "/home/yuhang/anaconda3/envs/paddle/lib/python3.10/site-packages/paddleocr/paddleocr.py", line 759, in __call__
res, _ = super().__call__(
File "/home/yuhang/anaconda3/envs/paddle/lib/python3.10/site-packages/paddleocr/ppstructure/predict_system.py", line 98, in __call__
cls_res = next(cls_result)
File "/home/yuhang/anaconda3/envs/paddle/lib/python3.10/site-packages/paddleclas-0.0.0-py3.10.egg/paddleclas/paddleclas.py", line 658, in predict_cls
yield self.predictor.predict(input_data)
File "/home/yuhang/anaconda3/envs/paddle/lib/python3.10/site-packages/paddleclas-0.0.0-py3.10.egg/paddleclas/deploy/python/predict_cls.py", line 95, in predict
self.predictor.run()
OSError: In user code:

File "tools/export_model.py", line 34, in <module>
  engine.export()
File "/paddle/PaddleClas/ppcls/engine/engine.py", line 492, in export
  paddle.jit.save(model, save_path)
File "/usr/local/python3.7.0/lib/python3.7/site-packages/decorator.py", line 232, in fun
  return caller(func, *(extras + args), **kw)
File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddle/fluid/wrapped_decorator.py", line 25, in __impl__
  return wrapped_func(*args, **kwargs)
File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddle/fluid/dygraph/base.py", line 51, in __impl__
  return func(*args, **kwargs)
File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddle/fluid/dygraph/jit.py", line 744, in save
  inner_input_spec)
File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py", line 517, in concrete_program_specify_input_spec
  *desired_input_spec)
File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py", line 427, in get_concrete_program
  concrete_program, partial_program_layer = self._program_cache[cache_key]
File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py", line 723, in __getitem__
  self._caches[item] = self._build_once(item)
File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py", line 714, in _build_once
  **cache_key.kwargs)
File "/usr/local/python3.7.0/lib/python3.7/site-packages/decorator.py", line 232, in fun
  return caller(func, *(extras + args), **kw)
File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddle/fluid/wrapped_decorator.py", line 25, in __impl__
  return wrapped_func(*args, **kwargs)
File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddle/fluid/dygraph/base.py", line 51, in __impl__
  return func(*args, **kwargs)
File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py", line 662, in from_func_spec
  outputs = static_func(*inputs)
File "/paddle/PaddleClas/ppcls/engine/engine.py", line 531, in forward
  x = self.base_model(x)
File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 917, in __call__
  return self._dygraph_call_func(*inputs, **kwargs)
File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 907, in _dygraph_call_func
  outputs = self.forward(*inputs, **kwargs)
File "/paddle/PaddleClas/ppcls/arch/backbone/legendary_models/pp_lcnet.py", line 293, in forward
  x = self.conv1(x)
File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 917, in __call__
  return self._dygraph_call_func(*inputs, **kwargs)
File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 907, in _dygraph_call_func
  outputs = self.forward(*inputs, **kwargs)
File "/paddle/PaddleClas/ppcls/arch/backbone/legendary_models/pp_lcnet.py", line 107, in forward
  x = self.conv(x)
File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 917, in __call__
  return self._dygraph_call_func(*inputs, **kwargs)
File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 907, in _dygraph_call_func
  outputs = self.forward(*inputs, **kwargs)
File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddle/nn/layer/conv.py", line 677, in forward
  use_cudnn=self._use_cudnn)
File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddle/nn/functional/conv.py", line 148, in _conv_nd
  type=op_type, inputs=inputs, outputs=outputs, attrs=attrs)
File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddle/fluid/layer_helper.py", line 43, in append_op
  return self.main_program.current_block().append_op(*args, **kwargs)
File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddle/fluid/framework.py", line 3184, in append_op
  attrs=kwargs.get("attrs", None))
File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddle/fluid/framework.py", line 2224, in __init__
  for frame in traceback.extract_stack():

ExternalError: CUDNN error(1), CUDNN_STATUS_NOT_INITIALIZED. 
  [Hint: 'CUDNN_STATUS_NOT_INITIALIZED'.  The cuDNN library was not initialized properly. This error is usually returned when a call to cudnnCreate() fails or when cudnnCreate() has not been called prior to calling another cuDNN routine. In the former case, it is usually due to an error in the CUDA Runtime API called by cudnnCreate() or by an error in the hardware setup.  ] (at /paddle/paddle/phi/backends/gpu/gpu_resources.cc:308)
  [operator < fused_conv2d_add_act > error]

我们提供了AceIssueSolver来帮助你解答问题，你是否想要它来解答(请填写yes/no)?/We provide AceIssueSolver to solve issues, do you want it? (Please write yes/no):

请尽量不要包含图片在问题中/Please try to not include the image in the issue.

PaddlePaddle / PaddleOCR

CUDNN_STATUS_NOT_INITIALIZED #11603