PaddlePaddle / PaddleDetection

Object Detection toolkit based on PaddlePaddle. It supports object detection, instance segmentation, multiple object tracking and real-time multi-person keypoint detection.
Apache License 2.0
12.79k stars 2.89k forks source link

conda 安装了cudnn7.6.5不知为啥还是显示7.2 并且报错了 #8860

Closed dianjingshilezhi closed 8 months ago

dianjingshilezhi commented 8 months ago

问题描述 Please describe your issue

conda 安装了cudnn7.6.5不知为啥还是显示7.2 并且报错了,如图

dianjingshilezhi commented 8 months ago

(paddle)@g5500-p40-1:~/code/paddle$ python tools/train.py -c configs/mask_rtdetr/mask_rtdetr_hgnetv2_l_6x_coco.yml --amp --eval anaconda3/envs/paddle/lib/python3.8/site-packages/setuptools/sandbox.py:13: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html import pkg_resources /anaconda3/envs/paddle/lib/python3.8/site-packages/pkg_resources/init.py:2871: DeprecationWarning: Deprecated call to pkg_resources.declare_namespace('mpl_toolkits'). Implementing implicit namespace packages (as specified in PEP 420) is preferred to pkg_resources.declare_namespace. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages declare_namespace(pkg) envs/paddle/lib/python3.8/site-packages/pkg_resources/init.py:2871: DeprecationWarning: Deprecated call to pkg_resources.declare_namespace('google'). Implementing implicit namespace packages (as specified in PEP 420) is preferred to pkg_resources.declare_namespace. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages declare_namespace(pkg) Warning: Unable to use numba in PP-Tracking, please install numba, for example(python3.7): pip install numba==0.56.4 Warning: Unable to use numba in PP-Tracking, please install numba, for example(python3.7): pip install numba==0.56.4 Warning: import ppdet from source directory without installing, run 'python setup.py install' to install ppdet firstly loading annotations into memory... Done (t=0.02s) creating index... index created! [03/09 13:47:25] ppdet.data.source.coco INFO: Load [720 samples valid, 0 samples invalid] in file /home/zhaoyu/dataset/Pothole_Segmentation_YOLOv8/train/annotations/result.json. W0309 13:47:25.489414 23452 gpu_resources.cc:61] Please NOTE: device: 0, GPU Compute Capability: 6.1, Driver API Version: 11.3, Runtime API Version: 10.2 W0309 13:47:25.493057 23452 gpu_resources.cc:91] device: 0, cuDNN Version: 7.2. [03/09 13:47:30] ppdet.utils.checkpoint INFO: ['fc.bias', 'fc.weight', 'last_conv.weight'] in pretrained weight is not used in the model, and its will not be loaded [03/09 13:47:30] ppdet.utils.checkpoint INFO: Finish loading model weights: /home/zhaoyu/.cache/paddle/weights//PPHGNetV2_L_ssld_pretrained.pdparams W0309 13:47:30.573891 23452 gpu_resources.cc:201] WARNING: device: . The installed Paddle is compiled with CUDNN 7.6, but CUDNN version in your machine is 7.2, which may cause serious incompatible bug. Please recompile or reinstall Paddle with compatible CUDNN version. Traceback (most recent call last): File "tools/train.py", line 209, in main() File "tools/train.py", line 205, in main run(FLAGS, cfg) File "tools/train.py", line 158, in run trainer.train(FLAGS.eval) File "/home/zhaoyu/code/paddle/ppdet/engine/trainer.py", line 559, in train outputs = model(data) File "/home/zhaoyu/anaconda3/envs/paddle/lib/python3.8/site-packages/paddle/fluid/dygraph/layers.py", line 930, in call return self._dygraph_call_func(*inputs, kwargs) File "/home/zhaoyu/anaconda3/envs/paddle/lib/python3.8/site-packages/paddle/fluid/dygraph/layers.py", line 915, in _dygraph_call_func outputs = self.forward(*inputs, *kwargs) File "/home/zhaoyu/code/paddle/ppdet/modeling/architectures/meta_arch.py", line 60, in forward out = self.get_loss() File "/home/zhaoyu/code/paddle/ppdet/modeling/architectures/detr.py", line 115, in get_loss return self._forward() File "/home/zhaoyu/code/paddle/ppdet/modeling/architectures/detr.py", line 81, in _forward body_feats = self.backbone(self.inputs) File "/home/zhaoyu/anaconda3/envs/paddle/lib/python3.8/site-packages/paddle/fluid/dygraph/layers.py", line 930, in call return self._dygraph_call_func(inputs, kwargs) File "/home/zhaoyu/anaconda3/envs/paddle/lib/python3.8/site-packages/paddle/fluid/dygraph/layers.py", line 915, in _dygraph_call_func outputs = self.forward(*inputs, kwargs) File "/home/zhaoyu/code/paddle/ppdet/modeling/backbones/hgnet_v2.py", line 451, in forward x = self.stem(x) File "/home/zhaoyu/anaconda3/envs/paddle/lib/python3.8/site-packages/paddle/fluid/dygraph/layers.py", line 930, in call return self._dygraph_call_func(*inputs, *kwargs) File "/home/zhaoyu/anaconda3/envs/paddle/lib/python3.8/site-packages/paddle/fluid/dygraph/layers.py", line 915, in _dygraph_call_func outputs = self.forward(inputs, kwargs) File "/home/zhaoyu/code/paddle/ppdet/modeling/backbones/hgnet_v2.py", line 183, in forward x = self.stem1(x) File "/home/zhaoyu/anaconda3/envs/paddle/lib/python3.8/site-packages/paddle/fluid/dygraph/layers.py", line 930, in call return self._dygraph_call_func(*inputs, kwargs) File "/home/zhaoyu/anaconda3/envs/paddle/lib/python3.8/site-packages/paddle/fluid/dygraph/layers.py", line 915, in _dygraph_call_func outputs = self.forward(*inputs, *kwargs) File "/home/zhaoyu/code/paddle/ppdet/modeling/backbones/hgnet_v2.py", line 94, in forward x = self.bn(x) File "/home/zhaoyu/anaconda3/envs/paddle/lib/python3.8/site-packages/paddle/fluid/dygraph/layers.py", line 930, in call return self._dygraph_call_func(inputs, kwargs) File "/home/zhaoyu/anaconda3/envs/paddle/lib/python3.8/site-packages/paddle/fluid/dygraph/layers.py", line 915, in _dygraph_call_func outputs = self.forward(*inputs, **kwargs) File "/home/zhaoyu/anaconda3/envs/paddle/lib/python3.8/site-packages/paddle/nn/layer/norm.py", line 656, in forward return batch_norm( File "/home/zhaoyu/anaconda3/envs/paddle/lib/python3.8/site-packages/paddle/nn/functional/norm.py", line 204, in batch_norm batch_normout, , , , , = _C_ops.batch_norm( OSError: (External) CUDNN error(3), CUDNN_STATUS_BAD_PARAM. [Hint: 'CUDNN_STATUS_BAD_PARAM'. An incorrect value or parameter was passed to the function. To correct, ensure that all the parameters being passed have valid values. ] (at /paddle/paddle/phi/kernels/gpu/batch_norm_kernel.cu:424) [operator < batch_norm > error]