search

AlexeyAB / darknet

YOLOv4 / Scaled-YOLOv4 / YOLO - Neural Networks for Object Detection (Windows and Linux version of Darknet )
http://pjreddie.com/darknet/
Other
21.76k stars 7.96k forks source link

Functions no longer supported with latest cuDNN #5905

Closed ptkahl closed 3 years ago

ptkahl commented 4 years ago

FILE: src/convolution_layer.c LINES: 304, 312, and 320 (in version prior to 10 July 2020) ISSUE: With Nvidia's recent release of CUDA 11.0, a corresponding cudNN 8.0 was released that does not include functions cudnnGetConvolutionForwardAlgorithm(), cudnnGetConvolutionBackwardDataAlgorithm(), and cudnnGetConvolutionBackwardFilterAlgorithm(), nor their respectively associated typdefs: cudnnConvolutionFwdPreference_t, cudnnConvolutionBwdDataPreference_t, and cudnnConvolutionBwdFilterPreference_t. (See Table 3 in https://docs.nvidia.com/deeplearning/sdk/cudnn-api/#api-changes.)

There may be similar cuDNN functions that can serve the intended purposes of these discontinued functions. For example, function cudnnGetConvolutionForwardAlgorithm_v7() is very similar to cudnnGetConvolutionForwardAlgorithm() in its description, but the parameters are not all the same.

Notes:

  1. AlexeyAB posted a fix on 10 July 2020 (https://github.com/AlexeyAB/darknet/commit/cb9148b38f8d2c5cec9ba717916268745235850c) but commented "cuDNN 8.0 supported (but works slow)" so it is not clear if this issue should be closed. The slowness may be due to cuDNN 8 performance rather than AlexeyAB's fix. The earlier cuDNN 8 versions were labeled as "Preview" versions and performance optimization was a noted issue. The Release Notes for cuDNN 8.0.3 indicate that most performance issues have been resolved. I recommend re-testing with the lastest versions of CUDA 11 and cuDNN 8 to see if AlexeyAB's fix still suffers from slow performance compared to previous code using CUDA 12 with cuDNN 7.

  2. AlexyAB posted another fix on 5 September 2020 (https://github.com/AlexeyAB/darknet/commit/eb0272f27acda1982fe4d30acd838fca427785a9) that is hoped to better address performance issues when using cuDNN 8. With this latest change, this issue should be closed.

Additional Notes:

Avnsx commented 4 years ago

Hey same issue here, I would strongly appreciate it if this case would be looked after and updated accordignly to newest cuda standards!

vfbsilva commented 4 years ago

I have the same issue with the following setup: community/cuda 11.1.0-2 [instalado] NVIDIA's GPU programming toolkit community/cudnn 8.0.4.30-1 [instalado] gcc (GCC) 10.2.0

on manjaro linux.

When compilling it fails it with:

gcc -Iinclude/ -I3rdparty/stb/include -DOPENCV `pkg-config --cflags opencv4` -DGPU -I/usr/local/cuda/include/ -DCUDNN -Wall -Wfatal-errors -Wno-unused-result -Wno-unknown-pragmas -fPIC -Ofast -DOPENCV -DGPU -DCUDNN -I/usr/local/cudnn/include -c ./src/convolutional_layer.c -o obj/convolutional_layer.o
./src/convolutional_layer.c: In function ‘cudnn_convolutional_setup’:
./src/convolutional_layer.c:284:24: error: ‘CUDNN_CONVOLUTION_FWD_PREFER_FASTEST’ undeclared (first use in this function); did you mean ‘CUDNN_CONVOLUTION_BWD_FILTER_ALGO_3’?
  284 |     int forward_algo = CUDNN_CONVOLUTION_FWD_PREFER_FASTEST;
      |                        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

I'm not familiar with the insides of the project so I wonder is there a work around this issue?

vfbsilva commented 3 years ago

Bump, any news regarding this issue or a work around?

ptkahl commented 3 years ago

I believe this issue was successfully addressed about half-a-year ago. I have only tested the "fix" on my system, but it seems to work. I updated the issue text some time ago, suggesting that it should be closed.

Sincerely, Patrick Kahl

On Tue, Jan 5, 2021 at 1:33 PM vfbsilva notifications@github.com wrote:

Bump, any news regarding this issue or a work around?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/AlexeyAB/darknet/issues/5905#issuecomment-754819081, or unsubscribe https://github.com/notifications/unsubscribe-auth/AO5LRTAWCPUNAXQQTVQZGCDSYNLOJANCNFSM4NZR3WWA .

AlexeyAB commented 3 years ago

Issue partially resolved on July 10, 2020, fully resolved on November 10, 2020 (when cuDNN 8.0.5 was released). So it seems the bug was in nVidia library cuDNN 8.0.4.

It works well with this configurations without any issue:

image image