opencv / opencv

Open Source Computer Vision Library
https://opencv.org
Apache License 2.0
75.95k stars 55.62k forks source link

Refactor DNN module to build with cudnn 9 #25412

Open ZelboK opened 1 month ago

ZelboK commented 1 month ago

A lot of APIs that are currently being used in the dnn module have been removed in cudnn 9. They were deprecated in 8. This PR updates said code accordingly to the newer API.

Some key notes: 1) This is my first PR. I am new to openCV. 2) opencv_test_core tests pass 3) On a 3080, cuda 12.4(should be irrelevant since I didn't build the opencv_modules, gcc 11.4, WSL 2. 4) For brevity I will avoid including macro code that will allow for older versions of cudnn to build.

I was unable to get the tests working for opencv_test_dnn and opencv_perf_dnn. The errors I get are of the following:

 OpenCV tests: Can't find required data file: dnn/onnx/conformance/node/test_reduce_prod_default_axes_keepdims_example/model.onnx in function 'findData'
" thrown in the test body.

So before I spend more time investigating I was hoping to get a maintainer to point me in the right direction here. I would like to run these tests and confirm things are working as intended. I may have missed some details.

Pull Request Readiness Checklist

relevant issue (https://github.com/opencv/opencv/issues/24983

ZelboK commented 1 month ago

I ran the precommit hook, but it is highlighting a lot of formatting related problems. Is there a simple fix for this?

asmorkalov commented 1 month ago

@ZelboK thanks for the patch. General comments:

ZelboK commented 1 month ago

@asmorkalov opencv_test_dnn opencv_test_core opencv_perf_dnn opencv_sanity_dnn

All pass. Updated to remove the formatting problems.

Also, is there no formatting hook I could run to do all the formatting automatically for me?

asmorkalov commented 1 month ago

The PR breaks CuDNN 8 (v 8.7.0, Ubuntu 18.04):

``` [ 72%] Building CXX object modules/dnn/CMakeFiles/opencv_dnn.dir/src/layers/const_layer.cpp.o In file included from /home/alexander/Projects/OpenCV/opencv-master/modules/dnn/src/layers/../cuda4dnn/csl/cudnn.hpp:8:0, from /home/alexander/Projects/OpenCV/opencv-master/modules/dnn/src/layers/../op_cuda.hpp:12, from /home/alexander/Projects/OpenCV/opencv-master/modules/dnn/src/layers/blank_layer.cpp:43: /home/alexander/Projects/OpenCV/opencv-master/modules/dnn/src/layers/../cuda4dnn/primitives/../csl/cudnn/recurrent.hpp: In constructor ‘cv::dnn::cuda4dnn::csl::cudnn::RNNDescriptor::RNNDescriptor(const cv::dnn::cuda4dnn::csl::cudnn::Handle&, cv::dnn::cuda4dnn::csl::cudnn::RNNDescriptor::RNNMode, int, int, bool, const cv::dnn::cuda4dnn::csl::cudnn::DropoutDescriptor&)’: /home/alexander/Projects/OpenCV/opencv-master/modules/dnn/src/layers/../cuda4dnn/primitives/../csl/cudnn/recurrent.hpp:129:11: error: ‘input_size’ was not declared in this scope input_size, hidden_size, ^ /home/alexander/Projects/OpenCV/opencv-master/modules/dnn/src/layers/../cuda4dnn/csl/cudnn/cudnn.hpp:22:53: note: in definition of macro ‘CUDA4DNN_CHECK_CUDNN’ ::cv::dnn::cuda4dnn::csl::cudnn::detail::check((call), CV_Func, __FILE__, __LINE__) ^~~~ /home/alexander/Projects/OpenCV/opencv-master/modules/dnn/src/layers/../cuda4dnn/primitives/../csl/cudnn/recurrent.hpp:129:11: note: suggested alternative: ‘hidden_size’ input_size, hidden_size, ^ /home/alexander/Projects/OpenCV/opencv-master/modules/dnn/src/layers/../cuda4dnn/csl/cudnn/cudnn.hpp:22:53: note: in definition of macro ‘CUDA4DNN_CHECK_CUDNN’ ::cv::dnn::cuda4dnn::csl::cudnn::detail::check((call), CV_Func, __FILE__, __LINE__) ^~~~ In file included from /home/alexander/Projects/OpenCV/opencv-master/modules/dnn/src/layers/../cuda4dnn/primitives/reshape.hpp:12:0, from /home/alexander/Projects/OpenCV/opencv-master/modules/dnn/src/layers/blank_layer.cpp:49: /home/alexander/Projects/OpenCV/opencv-master/modules/dnn/src/layers/../cuda4dnn/primitives/../csl/tensor_ops.hpp: In constructor ‘cv::dnn::cuda4dnn::csl::LSTM::LSTM(cv::dnn::cuda4dnn::csl::cudnn::Handle, const cv::dnn::cuda4dnn::csl::LSTM::params_type&)’: /home/alexander/Projects/OpenCV/opencv-master/modules/dnn/src/layers/../cuda4dnn/primitives/../csl/tensor_ops.hpp:548:13: error: ‘filterDesc’ was not declared in this scope filterDesc = FilterDescriptor(params.weights_shape); ^~~~~~~~~~ modules/dnn/CMakeFiles/opencv_dnn.dir/build.make:853: recipe for target 'modules/dnn/CMakeFiles/opencv_dnn.dir/src/layers/blank_layer.cpp.o' failed make[3]: *** [modules/dnn/CMakeFiles/opencv_dnn.dir/src/layers/blank_layer.cpp.o] Error 1 make[3]: *** Ожидание завершения заданий… In file included from /home/alexander/Projects/OpenCV/opencv-master/modules/dnn/src/layers/../cuda4dnn/csl/cudnn.hpp:8:0, from /home/alexander/Projects/OpenCV/opencv-master/modules/dnn/src/layers/../op_cuda.hpp:12, from /home/alexander/Projects/OpenCV/opencv-master/modules/dnn/src/layers/const_layer.cpp:10: /home/alexander/Projects/OpenCV/opencv-master/modules/dnn/src/layers/../cuda4dnn/primitives/../csl/cudnn/recurrent.hpp: In constructor ‘cv::dnn::cuda4dnn::csl::cudnn::RNNDescriptor::RNNDescriptor(const cv::dnn::cuda4dnn::csl::cudnn::Handle&, cv::dnn::cuda4dnn::csl::cudnn::RNNDescriptor::RNNMode, int, int, bool, const cv::dnn::cuda4dnn::csl::cudnn::DropoutDescriptor&)’: /home/alexander/Projects/OpenCV/opencv-master/modules/dnn/src/layers/../cuda4dnn/primitives/../csl/cudnn/recurrent.hpp:129:11: error: ‘input_size’ was not declared in this scope input_size, hidden_size, ^ /home/alexander/Projects/OpenCV/opencv-master/modules/dnn/src/layers/../cuda4dnn/csl/cudnn/cudnn.hpp:22:53: note: in definition of macro ‘CUDA4DNN_CHECK_CUDNN’ ::cv::dnn::cuda4dnn::csl::cudnn::detail::check((call), CV_Func, __FILE__, __LINE__) ^~~~ /home/alexander/Projects/OpenCV/opencv-master/modules/dnn/src/layers/../cuda4dnn/primitives/../csl/cudnn/recurrent.hpp:129:11: note: suggested alternative: ‘hidden_size’ input_size, hidden_size, ^ /home/alexander/Projects/OpenCV/opencv-master/modules/dnn/src/layers/../cuda4dnn/csl/cudnn/cudnn.hpp:22:53: note: in definition of macro ‘CUDA4DNN_CHECK_CUDNN’ ::cv::dnn::cuda4dnn::csl::cudnn::detail::check((call), CV_Func, __FILE__, __LINE__) ^~~~ In file included from /home/alexander/Projects/OpenCV/opencv-master/modules/dnn/src/layers/../cuda4dnn/primitives/const.hpp:12:0, from /home/alexander/Projects/OpenCV/opencv-master/modules/dnn/src/layers/const_layer.cpp:23: /home/alexander/Projects/OpenCV/opencv-master/modules/dnn/src/layers/../cuda4dnn/primitives/../csl/tensor_ops.hpp: In constructor ‘cv::dnn::cuda4dnn::csl::LSTM::LSTM(cv::dnn::cuda4dnn::csl::cudnn::Handle, const cv::dnn::cuda4dnn::csl::LSTM::params_type&)’: /home/alexander/Projects/OpenCV/opencv-master/modules/dnn/src/layers/../cuda4dnn/primitives/../csl/tensor_ops.hpp:548:13: error: ‘filterDesc’ was not declared in this scope filterDesc = FilterDescriptor(params.weights_shape); ^~~~~~~~~~ modules/dnn/CMakeFiles/opencv_dnn.dir/build.make:881: recipe for target 'modules/dnn/CMakeFiles/opencv_dnn.dir/src/layers/const_layer.cpp.o' failed ```
johnnynunez commented 2 weeks ago

The PR breaks CuDNN 8 (v 8.7.0, Ubuntu 18.04):

it was because the PR is incomplete. Some variables not exists as input_size. This modules need to refactored