search

dusty-nv / jetson-inference

Hello AI World guide to deploying deep-learning inference networks and deep vision primitives with TensorRT and NVIDIA Jetson.
https://developer.nvidia.com/embedded/twodaystoademo
MIT License
7.86k stars 2.98k forks source link

Failed to build CUDA engine #1889

Open osamaabdelatif opened 2 months ago

osamaabdelatif commented 2 months ago

Hi, I am trying to run object detection after building the project in my Jetson Nano,

detectnet --camera=csi://0

But i am getting below errors: [TRT] 1: [convolutionRunner.cpp::executeConv::511] Error Code 1: Cudnn (CUDNN_STATUS_EXECUTION_FAILED) [TRT] device GPU, failed to build CUDA engine [TRT] device GPU, failed to load networks/SSD-Mobilenet-v2/ssd_mobilenet_v2_coco.uff [TRT] detectNet -- failed to initialize. detectnet: failed to load detectNet model

My environment as below: Python 3.6.9

OS:

Linux ubuntu 4.9.337-tegra #1 SMP PREEMPT Tue Jun 11 16:07:05 PDT 2024 aarch64 aarch64 aarch64 GNU/Linux

nvcc --version

nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2021 NVIDIA Corporation Built on Sun_Feb_28_22:34:44_PST_2021 Cuda compilation tools, release 10.2, V10.2.300 Build cuda_10.2_r440.TC440_70.29663091_0

tensorrt version

Name: tensorrt Version: 8.2.1.9

nvidia-l4t-core

nvidia-l4t-core 32.7.5-20240611161210

highly appreciating any advice to fix or overcome this issue ! Thank you,

dusty-nv commented 2 months ago

Hi @osamaabdelatif, have you been able to run any of the other examples, like imagenet.py images/jellyfish.jpg ? I am wondering if you cuDNN installation is working ok.

osamaabdelatif commented 2 months ago

Hi @dusty-nv yea i got the same issue using imagenet.py images/jellyfish.jpg... How to check if cuDNN installed correctly, i used SDK Manager to flash the unit.

[TRT] 1: [convolutionRunner.cpp::executeConv::511] Error Code 1: Cudnn (CUDNN_STATUS_EXECUTION_FAILED) [TRT] device GPU, failed to build CUDA engine [TRT] device GPU, failed to load networks/Googlenet/bvlc_googlenet.caffemodel [TRT] failed to load networks/Googlenet/bvlc_googlenet.caffemodel [TRT] imageNet -- failed to initialize.

osamaabdelatif commented 2 months ago

Hi @dusty-nv I have below packages installed

ii libcudnn8 8.2.1.32-1+cuda10.2 arm64 cuDNN runtime libraries ii libcudnn8-dev 8.2.1.32-1+cuda10.2 arm64 cuDNN development libraries and headers ii libcudnn8-samples 8.2.1.32-1+cuda10.2 arm64 cuDNN documents and samples

and running cat /usr/include/cudnn_version.h | grep CUDNN_MAJOR -A 2 i get below output to check the cuDNN version

define CUDNN_MAJOR 8

define CUDNN_MINOR 2

define CUDNN_PATCHLEVEL 1

--

define CUDNN_VERSION (CUDNN_MAJOR 1000 + CUDNN_MINOR 100 + CUDNN_PATCHLEVEL)

endif / CUDNN_VERSION_H /