[Bug] find: 'release': No such file or directory find: 'CUDA Version': No such file or directory

Checklist

[x] I have searched related issues but cannot get the expected help.
[x] 2. I have read the FAQ documentation but cannot get the expected help.
[ ] 3. The bug has not been fixed in the latest version.

Describe the bug

C:\ProgramData\anaconda3\envs\openmmlab2\python.exe: can't open file 'tools/deploy.py': [Errno 2] No such file or directory (openmmlab2) PS D:\vwp\mmdeploy\build> cd .. (openmmlab2) PS D:\vwp\mmdeploy> python tools/deploy.py D:\vwp\mmdeploy\configs\mmdet\detection\detection_tensorrt_static-640x640.py D:\vwp\mmdeploy\trans_file\detection_config.py D:\vwp\mmdeploy\trans_file\best_coco_bbox_mAP_epoch_166.pth D:\vwp\mmdeploy\trans_file\val\000000003536.jpg --work-dir work_dir --show --device cuda:0 --dump-info 07/18 20:28:30 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized. 07/18 20:28:30 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "mmdet_tasks" registry tree. As a workaround, the current "mmdet_tasks" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized. 07/18 20:28:32 - mmengine - INFO - Start pipeline mmdeploy.apis.pytorch2onnx.torch2onnx in subprocess 07/18 20:28:32 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized. 07/18 20:28:32 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "mmdet_tasks" registry tree. As a workaround, the current "mmdet_tasks" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized. Loads checkpoint by local backend from path: D:\vwp\mmdeploy\trans_file\best_coco_bbox_mAP_epoch_166.pth 07/18 20:28:34 - mmengine - WARNING - DeprecationWarning: get_onnx_config will be deprecated in the future. 07/18 20:28:34 - mmengine - INFO - Export PyTorch model to ONNX: work_dir\end2end.onnx. 07/18 20:28:34 - mmengine - WARNING - Can not find torch._C._jit_pass_onnx_autograd_function_process, function rewrite will not be applied 07/18 20:28:34 - mmengine - WARNING - Can not find mmdet.models.utils.transformer.PatchMerging.forward, function rewrite will not be applied d:\vwp\mmdeploy\mmdeploy\codebase\mmdet\models\detectors\single_stage.py:84: TracerWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won't change the number of iterations executed (and might lead to errors or silently give incorrect results). img_shape = [int(val) for val in img_shape] d:\vwp\mmdeploy\mmdeploy\codebase\mmdet\models\detectors\single_stage.py:84: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! img_shape = [int(val) for val in img_shape] d:\vwp\mmdeploy\mmdeploy\core\optimizers\function_marker.py:160: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! ys_shape = tuple(int(s) for s in ys.shape) C:\ProgramData\anaconda3\envs\openmmlab2\lib\site-packages\torch\functional.py:478: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at C:\cb\pytorch_1000000000000\work\aten\src\ATen\native\TensorShape.cpp:2895.) return _VF.meshgrid(tensors, **kwargs) # type: ignore[attr-defined] d:\vwp\mmdeploy\mmdeploy\mmcv\ops\nms.py:451: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! int(scores.shape[-1]), d:\vwp\mmdeploy\mmdeploy\mmcv\ops\nms.py:148: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! out_boxes = min(num_boxes, after_topk) WARNING: The shape inference of mmdeploy::TRTBatchedNMS type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function. WARNING: The shape inference of mmdeploy::TRTBatchedNMS type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function. WARNING: The shape inference of mmdeploy::TRTBatchedNMS type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function. WARNING: The shape inference of mmdeploy::TRTBatchedNMS type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function. WARNING: The shape inference of mmdeploy::TRTBatchedNMS type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function. WARNING: The shape inference of mmdeploy::TRTBatchedNMS type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function. 07/18 20:28:39 - mmengine - INFO - Execute onnx optimize passes. 07/18 20:28:39 - mmengine - INFO - Finish pipeline mmdeploy.apis.pytorch2onnx.torch2onnx 07/18 20:28:42 - mmengine - INFO - Start pipeline mmdeploy.apis.utils.utils.to_backend in subprocess 07/18 20:28:42 - mmengine - INFO - Successfully loaded tensorrt plugins from d:\vwp\mmdeploy\mmdeploy\lib\mmdeploy_tensorrt_ops.dll [07/18/2023-20:28:43] [TRT] [I] [MemUsageChange] Init CUDA: CPU +291, GPU +0, now: CPU 12654, GPU 860 (MiB) [07/18/2023-20:28:46] [TRT] [I] [MemUsageChange] Init builder kernel library: CPU +877, GPU +172, now: CPU 14616, GPU 1032 (MiB) [07/18/2023-20:28:46] [TRT] [W] CUDA lazy loading is not enabled. Enabling it can significantly reduce device memory usage and speed up TensorRT initialization. See "Lazy Loading" section of CUDA documentation https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#lazy-loading [07/18/2023-20:28:46] [TRT] [I] ---------------------------------------------------------------- [07/18/2023-20:28:46] [TRT] [I] Input filename: work_dir\end2end.onnx [07/18/2023-20:28:46] [TRT] [I] ONNX IR version: 0.0.6 [07/18/2023-20:28:46] [TRT] [I] Opset version: 11 [07/18/2023-20:28:46] [TRT] [I] Producer name: pytorch [07/18/2023-20:28:46] [TRT] [I] Producer version: 1.12.1 [07/18/2023-20:28:46] [TRT] [I] Domain: [07/18/2023-20:28:46] [TRT] [I] Model version: 0 [07/18/2023-20:28:46] [TRT] [I] Doc string: [07/18/2023-20:28:46] [TRT] [I] ---------------------------------------------------------------- [07/18/2023-20:28:46] [TRT] [W] onnx2trt_utils.cpp:374: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32. [07/18/2023-20:28:46] [TRT] [I] No importer registered for op: TRTBatchedNMS. Attempting to import as plugin. [07/18/2023-20:28:46] [TRT] [I] Searching for plugin: TRTBatchedNMS, plugin_version: 1, plugin_namespace: [07/18/2023-20:28:46] [TRT] [I] Successfully created plugin: TRTBatchedNMS find: 'release': No such file or directory find: 'CUDA Version': No such file or directory [07/18/2023-20:28:46] [TRT] [I] BuilderFlag::kTF32 is set but hardware does not support TF32. Disabling TF32. [07/18/2023-20:28:46] [TRT] [I] Graph optimization time: 0.0375647 seconds. [07/18/2023-20:28:47] [TRT] [I] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +645, GPU +226, now: CPU 14329, GPU 1258 (MiB) [07/18/2023-20:28:47] [TRT] [I] [MemUsageChange] Init cuDNN: CPU +142, GPU +50, now: CPU 14471, GPU 1308 (MiB) [07/18/2023-20:28:47] [TRT] [W] TensorRT was linked against cuDNN 8.9.0 but loaded cuDNN 8.3.2 [07/18/2023-20:28:47] [TRT] [I] BuilderFlag::kTF32 is set but hardware does not support TF32. Disabling TF32. [07/18/2023-20:30:13] [TRT] [I] Detected 1 inputs and 2 output network tensors. [07/18/2023-20:30:13] [TRT] [I] Total Host Persistent Memory: 367104 [07/18/2023-20:30:13] [TRT] [I] Total Device Persistent Memory: 2822144 [07/18/2023-20:30:13] [TRT] [I] Total Scratch Memory: 2067712 [07/18/2023-20:30:13] [TRT] [I] [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 7 MiB, GPU 77 MiB [07/18/2023-20:30:13] [TRT] [I] [BlockAssignment] Started assigning block shifts. This will take 232 steps to complete. [07/18/2023-20:30:13] [TRT] [I] [BlockAssignment] Algorithm ShiftNTopDown took 11.6749ms to assign 10 blocks to 232 nodes requiring 28058624 bytes. [07/18/2023-20:30:13] [TRT] [I] Total Activation Memory: 28057600 [07/18/2023-20:30:13] [TRT] [I] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +8, now: CPU 14613, GPU 1358 (MiB) [07/18/2023-20:30:13] [TRT] [I] [MemUsageChange] Init cuDNN: CPU +0, GPU +10, now: CPU 14613, GPU 1368 (MiB) [07/18/2023-20:30:13] [TRT] [W] TensorRT was linked against cuDNN 8.9.0 but loaded cuDNN 8.3.2 [07/18/2023-20:30:13] [TRT] [I] [MemUsageChange] TensorRT-managed allocation in building engine: CPU +3, GPU +30, now: CPU 3, GPU 30 (MiB) 07/18 20:30:14 - mmengine - INFO - Finish pipeline mmdeploy.apis.utils.utils.to_backend 07/18 20:30:14 - mmengine - INFO - visualize tensorrt model start. 07/18 20:30:17 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized. 07/18 20:30:17 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "mmdet_tasks" registry tree. As a workaround, the current "mmdet_tasks" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized. 07/18 20:30:17 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "backend_detectors" registry tree. As a workaround, the current "backend_detectors" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized. 07/18 20:30:17 - mmengine - INFO - Successfully loaded tensorrt plugins from d:\vwp\mmdeploy\mmdeploy\lib\mmdeploy_tensorrt_ops.dll 07/18 20:30:17 - mmengine - INFO - Successfully loaded tensorrt plugins from d:\vwp\mmdeploy\mmdeploy\lib\mmdeploy_tensorrt_ops.dll [07/18/2023-20:30:18] [TRT] [W] TensorRT was linked against cuDNN 8.9.0 but loaded cuDNN 8.3.2 [07/18/2023-20:30:18] [TRT] [W] TensorRT was linked against cuDNN 8.9.0 but loaded cuDNN 8.3.2 [07/18/2023-20:30:18] [TRT] [W] CUDA lazy loading is not enabled. Enabling it can significantly reduce device memory usage and speed up TensorRT initialization. See "Lazy Loading" section of CUDA documentation https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#lazy-loading 07/18 20:30:22 - mmengine - INFO - visualize tensorrt model success. 07/18 20:30:22 - mmengine - INFO - visualize pytorch model start. 07/18 20:30:25 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized. 07/18 20:30:25 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "mmdet_tasks" registry tree. As a workaround, the current "mmdet_tasks" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized. Loads checkpoint by local backend from path: D:\vwp\mmdeploy\trans_file\best_coco_bbox_mAP_epoch_166.pth C:\ProgramData\anaconda3\envs\openmmlab2\lib\site-packages\torch\functional.py:478: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at C:\cb\pytorch_1000000000000\work\aten\src\ATen\native\TensorShape.cpp:2895.) return _VF.meshgrid(tensors, **kwargs) # type: ignore[attr-defined] 07/18 20:30:30 - mmengine - INFO - visualize pytorch model success. 07/18 20:30:30 - mmengine - INFO - All process success.

Reproduction

command: python tools/deploy.py D:\vwp\mmdeploy\configs\mmdet\detection\detection_tensorrt_static-640x640.py D:\vwp\mmdeploy\trans_file\detection_config.py D:\vwp\mmdeploy\trans_file\best_coco_bbox_mAP_epoch_166.pth D:\vwp\mmdeploy\trans_file\val\000000003536.jpg --work-dir work_dir --show --device cuda --dump-info

Environment

07/18 21:23:16 - mmengine - INFO -

07/18 21:23:16 - mmengine - INFO - **********Environmental information**********
07/18 21:23:19 - mmengine - INFO - sys.platform: win32
07/18 21:23:19 - mmengine - INFO - Python: 3.8.17 (default, Jul  5 2023, 20:44:21) [MSC v.1916 64 bit (AMD64)]
07/18 21:23:19 - mmengine - INFO - CUDA available: True
07/18 21:23:19 - mmengine - INFO - numpy_random_seed: 2147483648
07/18 21:23:19 - mmengine - INFO - GPU 0: NVIDIA T600 Laptop GPU
07/18 21:23:19 - mmengine - INFO - CUDA_HOME: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.6
07/18 21:23:19 - mmengine - INFO - NVCC: Cuda compilation tools, release 11.6, V11.6.55
07/18 21:23:19 - mmengine - INFO - MSVC: 用于 x64 的 Microsoft (R) C/C++ 优化编译器 19.29.30151 版
07/18 21:23:19 - mmengine - INFO - GCC: n/a
07/18 21:23:19 - mmengine - INFO - PyTorch: 1.12.1
07/18 21:23:19 - mmengine - INFO - PyTorch compiling details: PyTorch built with:
  - C++ Version: 199711
  - MSVC 192829337
  - Intel(R) Math Kernel Library Version 2020.0.2 Product Build 20200624 for Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v2.6.0 (Git Hash 52b5f107dd9cf10910aaa19cb47f3abf9b349815)
  - OpenMP 2019
  - LAPACK is enabled (usually provided by MKL)
  - CPU capability usage: AVX2
  - CUDA Runtime 11.6
  - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_37,code=compute_37
  - CuDNN 8.3.2  (built against CUDA 11.5)
  - Magma 2.5.4
  - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.6, CUDNN_VERSION=8.3.2, CXX_COMPILER=C:/cb/pytorch_1000000000000/work/tmp_bin/sccache-cl.exe, CXX_FLAGS=/DWIN32 /D_WINDOWS /GR /EHsc /w /bigobj -DUSE_PTHREADPOOL -openmp:experimental -IC:/cb/pytorch_1000000000000/work/mkl/include -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DUSE_FBGEMM -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.12.1, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=OFF, USE_MPI=OFF, USE_NCCL=OFF, USE_NNPACK=OFF, USE_OPENMP=ON, USE_ROCM=OFF,

07/18 21:23:19 - mmengine - INFO - TorchVision: 0.13.1
07/18 21:23:19 - mmengine - INFO - OpenCV: 4.8.0
07/18 21:23:19 - mmengine - INFO - MMEngine: 0.8.1
07/18 21:23:19 - mmengine - INFO - MMCV: 2.0.1
07/18 21:23:19 - mmengine - INFO - MMCV Compiler: MSVC 192930148
07/18 21:23:19 - mmengine - INFO - MMCV CUDA Compiler: 11.6
07/18 21:23:19 - mmengine - INFO - MMDeploy: 1.1.0+01f8dd4
07/18 21:23:19 - mmengine - INFO -

07/18 21:23:19 - mmengine - INFO - **********Backend information**********
07/18 21:23:19 - mmengine - INFO - tensorrt:    8.6.1
07/18 21:23:19 - mmengine - INFO - tensorrt custom ops: Available
07/18 21:23:19 - mmengine - INFO - ONNXRuntime: 1.15.1
07/18 21:23:19 - mmengine - INFO - ONNXRuntime-gpu:     None
07/18 21:23:19 - mmengine - INFO - ONNXRuntime custom ops:      NotAvailable
07/18 21:23:19 - mmengine - INFO - pplnn:       None
07/18 21:23:19 - mmengine - INFO - ncnn:        None
07/18 21:23:19 - mmengine - INFO - snpe:        None
07/18 21:23:19 - mmengine - INFO - openvino:    None
07/18 21:23:19 - mmengine - INFO - torchscript: 1.12.1
07/18 21:23:19 - mmengine - INFO - torchscript custom ops:      NotAvailable
07/18 21:23:19 - mmengine - INFO - rknn-toolkit:        None
07/18 21:23:19 - mmengine - INFO - rknn-toolkit2:       None
07/18 21:23:19 - mmengine - INFO - ascend:      None
07/18 21:23:19 - mmengine - INFO - coreml:      None
07/18 21:23:19 - mmengine - INFO - tvm: None
07/18 21:23:19 - mmengine - INFO - vacc:        None
07/18 21:23:19 - mmengine - INFO -

07/18 21:23:19 - mmengine - INFO - **********Codebase information**********
07/18 21:23:19 - mmengine - INFO - mmdet:       3.0.0
07/18 21:23:19 - mmengine - INFO - mmseg:       None
07/18 21:23:19 - mmengine - INFO - mmpretrain:  None
07/18 21:23:19 - mmengine - INFO - mmocr:       None
07/18 21:23:19 - mmengine - INFO - mmagic:      None
07/18 21:23:19 - mmengine - INFO - mmdet3d:     None
07/18 21:23:19 - mmengine - INFO - mmpose:      1.0.0
07/18 21:23:19 - mmengine - INFO - mmrotate:    None
07/18 21:23:19 - mmengine - INFO - mmaction:    None
07/18 21:23:19 - mmengine - INFO - mmrazor:     None

Error traceback

No response

open-mmlab / mmdeploy