shouxieai / tensorRT_Pro

C++ library based on tensorrt integration
MIT License
2.56k stars 544 forks source link

][trt_builder.cpp:36]:NVInfer: TensorRT was linked against cuBLAS/cuBLAS LT 11.6.3 but loaded cuBLAS/cuBLAS LT 11.3.1 #49

Closed sainttelant closed 2 years ago

sainttelant commented 2 years ago

warning, errors are showed above, it is annoying and weird to concern the compatibility issues among tensorrt version and cuda, and cuda toolkit versions. i cannot figure out the difference among them, any help will be approciated!!!

i have installed tensorrt !!!

dpkg -l | grep tensorrt ii nv-tensorrt-repo-ubuntu1804-cuda10.0-trt5.0.2.6-ga-20181009 1-1 amd64 nv-tensorrt repository configuration files ii nv-tensorrt-repo-ubuntu1804-cuda10.1-trt5.1.5.0-ga-20190427 1-1 amd64 nv-tensorrt repository configuration files ii nv-tensorrt-repo-ubuntu1804-cuda11.4-trt8.2.1.8-ga-20211117 1-1 amd64 nv-tensorrt repository configuration files ii tensorrt 8.2.1.8-1+cuda11.4 amd64

it should be 8.2.18 cuda11.4 as i am concerned.

and after i typed nvcc -V , the cuda version is nvcc as follows:

NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2020 NVIDIA Corporation Built on Mon_Nov_30_19:08:53_PST_2020 Cuda compilation tools, release 11.2, V11.2.67 Build cuda_11.2.r11.2/compiler.29373293_0

so which version of toolkit should i install??? currently, my cuda toolkit is v11.5.0 i think

sainttelant commented 2 years ago

i 've tried to reduce the cudatoolkit version, and reinstalled all nvidia relevant drivers and tensorrt etc. and my current toolkit is compatability of cuda 11.2 version now ,however the error is same.......

i compile the centernet in ubuntu18.04, the above system, it throws the errors::

[ 19%] Linking CXX executable ../workspace/pro [100%] Built target pro [2021-12-27 16:16:04][info][app_centernet.cpp:90]:===================== test FP32 centernet_r18_dcn ================================== [2021-12-27 16:16:05][warn][trt_builder.cpp:36]:NVInfer: TensorRT was linked against cuBLAS/cuBLAS LT 11.6.3 but loaded cuBLAS/cuBLAS LT 11.3.1 [2021-12-27 16:16:05][warn][trt_builder.cpp:36]:NVInfer: TensorRT was linked against cuBLAS/cuBLAS LT 11.6.3 but loaded cuBLAS/cuBLAS LT 11.3.1 [2021-12-27 16:16:05][info][trt_infer.cpp:177]:Infer 0x7f5254000c50 detail [2021-12-27 16:16:05][info][trt_infer.cpp:178]: Max Batch Size: 16 [2021-12-27 16:16:05][info][trt_infer.cpp:179]: Inputs: 1 [2021-12-27 16:16:05][info][trt_infer.cpp:183]: 0.images : shape {16 x 3 x 512 x 512}, Float32 [2021-12-27 16:16:05][info][trt_infer.cpp:186]: Outputs: 1 [2021-12-27 16:16:05][info][trt_infer.cpp:190]: 0.output : shape {16 x 16384 x 164}, Float32 files' size:0 Segmentation fault (core dumped) CMakeFiles/centernet.dir/build.make:70: recipe for target 'CMakeFiles/centernet' failed make[3]: [CMakeFiles/centernet] Error 139 CMakeFiles/Makefile2:203: recipe for target 'CMakeFiles/centernet.dir/all' failed make[2]: [CMakeFiles/centernet.dir/all] Error 2 CMakeFiles/Makefile2:210: recipe for target 'CMakeFiles/centernet.dir/rule' failed make[1]: [CMakeFiles/centernet.dir/rule] Error 2 Makefile:176: recipe for target 'centernet' failed make: [centernet] Error 2 '

oh my, i 've no idea how to solve it ,,,,,

sainttelant commented 2 years ago

@shouxieai

sainttelant commented 2 years ago

i found this one is warning, it can run after i input correct path of imgs files, however,

2021-12-27 18:05:32][error][preprocess_kernel.cu:376]:launch failed: no kernel image is available for execution on the device [2021-12-27 18:05:33][error][DCNv2.cu:304]:launch failed: no kernel image is available for execution on the device [2021-12-27 18:05:33][error][DCNv2.cu:315]:launch failed: no kernel image is available for execution on the device [2021-12-27 18:05:33][error][DCNv2.cu:329]:launch failed: no kernel image is available for execution on the device [2021-12-27 18:05:33][error][DCNv2.cu:304]:launch failed: no kernel image is available for execution on the device [2021-12-27 18:05:33][error][DCNv2.cu:315]:launch failed: no kernel image is available for execution on the device [2021-12-27 18:05:33][error][DCNv2.cu:329]:launch failed: no kernel image is available for execution on the device [2021-12-27 18:05:33][error][DCNv2.cu:304]:launch failed: no kernel image is available for execution on the device [2021-12-27 18:05:33][error][DCNv2.cu:315]:launch failed: no kernel image is available for execution on the device [2021-12-27 18:05:33][error][DCNv2.cu:329]:launch failed: no kernel image is available for execution on the device [2021-12-27 18:05:33][error][centernet_decode.cu:175]:launch failed: no kernel image is available for execution on the device [2021-12-27 18:05:33][error][centernet_decode.cu:179]:launch failed: no kernel image is available for execution on the device [2021-12-27 18:05:33][error][preprocess_kernel.cu:376]:launch failed: no kernel image is available for execution on the device [2021-12-27 18:05:33][error][preprocess_kernel.cu:376]:launch failed: no kernel image is available for execution on the device [2021-12-27 18:05:33][error][preprocess_kernel.cu:376]:launch failed: no kernel image is available for execution on the device [2021-12-27 18:05:33][error][preprocess_kernel.cu:376]:launch failed: no kernel image is available for execution on the device [2021-12-27 18:05:33][error][preprocess_kernel.cu:376]:launch failed: no kernel image is available for execution on the device [2021-12-27 18:05:33][error][preprocess_kernel.cu:376]:launch failed: no kernel image is available for execution on the device

sainttelant commented 2 years ago

set -gencode=arch=compute_85,code=sm_85 in cmakelists for rtx3090 , it works