inference on GPU: Missing version information + Error on running demo.py

[zybermonk]

I currently have the following dependencies which closely align with the requirements.txt

• Python: 3.8.9
• RTX 3090 with cuda 12.2 global driver
• I have the following installed inside my conda environment nvidia-cublas-cu11 2022.4.8 pypi_0 pypi nvidia-cublas-cu117 11.10.1.25 pypi_0 pypi nvidia-cuda-runtime-cu11 2022.4.25 pypi_0 pypi nvidia-cudnn-cu11 2022.5.19 pypi_0 pypi nvidia-cudnn-cu113 8.2.1.32 pypi_0 pypi nvidia-cudnn-cu116 8.4.0.27 pypi_0 pypi
• Other requirements: pycuda==2021.1 onnx==1.12.0 onnxruntime==1.12.0 opencv-python==4.5.4.60 torch==1.11.0+cu113 torchvision==0.12.0+cu113 torchaudio==0.11.0

However, It would be beneficial to have some clarifications regarding the recommended CUDA, for all the other subdependencies. And please help how to fix the following error:

(tool) home/xxxxxx$ python demo.py
[2024-07-30 12:27:43] [INFO] [Pycuda] Cuda Version: (11, 2, 0)
[2024-07-30 12:27:43] [INFO] [Driver] Cuda Version: 12020
[2024-07-30 12:27:43] [INFO] ----------------------------------------
[2024-07-30 12:27:43] [INFO] Detector Model Type : UFLDV2_TUSIMPLE
[2024-07-30 12:27:44] [INFO] UfldDetectorV2 Type : [trt] || Version : [CUDAExecutionProvider]
[2024-07-30 12:27:44] [INFO] -> Input Shape : (1, 3, 320, 800)
[2024-07-30 12:27:44] [INFO] -> Input Type  : <class 'numpy.float32'>
[2024-07-30 12:27:44] [INFO] -> Output Shape : [(1, 100, 56, 4), (1, 100, 41, 4), (1, 2, 56, 4), (1, 2, 41, 4)]
[2024-07-30 12:27:44] [INFO] ObjectDetector Model Type : YOLOV8
[2024-07-30 12:27:44] [INFO] YoloDetector Type : [trt] || Version : [CUDAExecutionProvider]
[2024-07-30 12:27:44] [INFO] -> Input Shape : (1, 3, 480, 640)
[2024-07-30 12:27:44] [INFO] -> Input Type  : <class 'numpy.float32'>
[2024-07-30 12:27:44] [INFO] -> Output Shape : [(1, 84, 6300)]
-------------------------------------------------------------------
PyCUDA ERROR: The context stack was not empty upon module cleanup.
-------------------------------------------------------------------
A context was still active when the context stack was being
cleaned up. At this point in our execution, CUDA may already
have been deinitialized, so there is no way we can finish
cleanly. The program will be aborted now.
Use Context.pop() to avoid this problem.
-------------------------------------------------------------------
Aborted (core dumped)

I also ran the quantized versions too, but still the same error. I tried various fixes which led nowhere. @jason-li-831202 Please point me in the right direction.

Thank you

[jason-li-831202]

What is your tensorrt version?

[zybermonk]

Hi,

I have an update. I got it working.

So, The following are the requirements that work out of the box, given the right numpy version ( 1.23.0, which I found most compatible with python3.8 and the packages in this repo.) is installed before installing the requirements.txt

@jason-li-831202, Thanks for the response, and w.r.t tensorrt, the most compatible version was found to be 8.5.3.1, so it is good to have it already in the requirements file

addict
numba
lap
scipy
pycuda   #Latest version also works
tensorrt==8.5.3.1
torchsummary
onnx==1.12.0
onnxruntime==1.12.0
opencv-python==4.5.4.60
torch==1.11.0+cu113 
torchvision==0.12.0+cu113 
torchaudio==0.11.0 
--extra-index-url https://download.pytorch.org/whl/cu113