Open elias-work opened 3 years ago
I have tested on an RTX 2080 Ti with CUDA 10.2 and Ubuntu 18.04 now too with the same issue.
It turns out the issue was because I had parallelized ZED camera reading and detection to improve performance on one streaming source. Apparently the CUDA calls are unsafe in this context.
Is this predictable, or does it suggest a bug, perhaps in ZED SDK?
I also encountered the same problem. Has anyone solved it?
same issue here when running yolov4-tiny on Xavier-NX, compiling with:
GPU=1 CUDNN=1 CUDNN_HALF=1 OPENCV=1 AVX=0 OPENMP=0 LIBSO=1 ZED_CAMERA=1 ZED_CAMERA_v2_8=0
Has anyone solved it?
OS Ubuntu 20.04 CUDA 11.2 CUDNN 8.1.1 GPU Quadro T2000 (capability 7.5) darknet interface Python 3.8 (darknet.py) Model YOLOv4 custom trained, 416x416 Make options
In the project I am working on, we can run YOLOv4 on a video file (read via opencv) and this works very well. I am attempting to run YOLOv4 on a ZED recording read via ZED SDK. Unfortunately with it loaded I get this strange error in the very first
darknet.py:detect_image
call:Setting CUDNN=0 changes the error to:
(related? https://github.com/stereolabs/zed-yolo/issues/29)
I can comment the
detect_image
call and it will keep reading video correctly and never crash. Note, we are passing (416,416,3) data todetect_image
and I have made sure it is correct when it crashes.When it crashes I still have >1GB video memory free.
So I am not 100% certain this is a darknet issue, but since my opaque error happens in darknet, does anyone have any advice for things to try? I have exhausted my list of reasonable things to try, I would like to test on a better GPU but am unable to for right now. I have not tested CUDNN_HALF=1 because I cannot use it in general as my GPU lacks tensor cores.