search

jkjung-avt / tensorrt_demos

TensorRT MODNet, YOLOv4, YOLOv3, SSD, MTCNN, and GoogLeNet
https://jkjung-avt.github.io/
MIT License
1.74k stars 545 forks source link

Error in onnx to tensorrt. cuda failure in file 'yolo_layer.cu' line 51:35 Aborted (core dumped) Request assistance from the big guys #547

Closed wangyu96-art closed 2 years ago

wangyu96-art commented 2 years ago

Error in onnx to tensorrt. cuda failure in file 'yolo_layer.cu' line 51:35 Aborted (core dumped) Request assistance from the big guys

wangyu96-art commented 2 years ago

Using a custom yolov3 model after cropping

wangyu96-art commented 2 years ago

CHECK(cudaMalloc(&mAnchors, MAX_ANCHORS 2 sizeof(float)));

jkjung-avt commented 2 years ago

Please provide your cfg file. I will take a look.

wangyu96-art commented 2 years ago

converted.zip

jkjung-avt commented 2 years ago

Your cfg file looks OK to me.

Failure to execute this line of code means the program cannot allocate memory on the GPU.

CHECK(cudaMalloc(&mAnchors, MAX_ANCHORS * 2 * sizeof(float)));

Which GPU are you using? Do you have multiple GPUs on the system? If so, try to set, say, CUDA_VISIBLE_DEVICES=0 (environment variable) to see if it helps.

wangyu96-art commented 2 years ago

I'm using a Jetson nano development board, I'll try your help first, thanks.

jkjung-avt commented 2 years ago

There should be only 1 GPU on your Jetson DevKit...

I wonder whether there's a hardware problem on your Jetson, or maybe most of the system memories have been occupied by other programs.