opencv / opencv

Open Source Computer Vision Library
https://opencv.org
Apache License 2.0
77.21k stars 55.69k forks source link

DNN with CUDA not working (Restarting Core in Spyder) #22379

Open IltonWhatever opened 1 year ago

IltonWhatever commented 1 year ago

OpenCV => 4.6 Operating System / Platform => Windows It was too slow to compute DNN with cpu. So I built and checked all of these cmake options to use the GPU.

net.setPreferableBackend(cv2.dnn.DNN_BACKEND_CUDA)
net.setPreferableTarget(cv2.dnn.DNN_TARGET_CUDA)

In the above command, there is no error with CUDA , but the kernel restart when compiling...

Selecting Windows SDK version 10.0.19041.0 to target Windows 10.0.19043.
ocv_init_download: OpenCV source tree is not fetched as git repository. 3rdparty resources will be downloaded from github.com by default.
Detected processor: AMD64
Could NOT find PythonInterp (missing: PYTHON_EXECUTABLE) (Required is at least version "2.7")
libjpeg-turbo: VERSION = 2.1.2, BUILD = opencv-4.6.0-libjpeg-turbo
Could NOT find OpenJPEG (minimal suitable version: 2.0, recommended version >= 2.3.1). OpenJPEG will be built from sources
OpenJPEG: VERSION = 2.4.0, BUILD = opencv-4.6.0-openjp2-2.4.0
OpenJPEG libraries will be built from sources: libopenjp2 (version "2.4.0")
found Intel IPP (ICV version): 2020.0.0 [2020.0.0 Gold]
at: Z:/IA CAI/build_nova/3rdparty/ippicv/ippicv_win/icv
found Intel IPP Integration Wrappers sources: 2020.0.0
at: Z:/IA CAI/build_nova/3rdparty/ippicv/ippicv_win/iw
CUDA detected: 11.7
CUDA: Using CUDA_ARCH_BIN=5.0
CUDA NVCC target flags: -gencode;arch=compute_50,code=sm_50;-D_FORCE_INLINES
CUDA: MSVS generator is detected. Disabling CMake re-run checks (CMAKE_SUPPRESS_REGENERATION=ON). You need to run CMake manually if updates are required.
Could not find OpenBLAS include. Turning OpenBLAS_FOUND off
Could not find OpenBLAS lib. Turning OpenBLAS_FOUND off
Could NOT find BLAS (missing: BLAS_LIBRARIES) 
Could NOT find LAPACK (missing: LAPACK_LIBRARIES) 
    Reason given by package: LAPACK could not be found because dependency BLAS could not be found.

Could NOT find JNI (missing: JAVA_INCLUDE_PATH JAVA_INCLUDE_PATH2 AWT JVM) 
VTK is not found. Please set -DVTK_DIR in CMake to VTK build directory, or to VTK install subdirectory with VTKConfig.cmake file
Module opencv_alphamat disabled because the following dependencies are not found: Eigen
freetype2:   NO
harfbuzz:    NO
Julia not found. Not compiling Julia Bindings. 
Module opencv_ovis disabled because OGRE3D was not found
No preference for use of exported gflags CMake configuration set, and no hints for include/library directories provided. Defaulting to preferring an installed/exported gflags CMake configuration if available.
Failed to find installed gflags CMake configuration, searching for gflags build directories exported with CMake.
Failed to find gflags - Failed to find an installed/exported CMake configuration for gflags, will perform search for installed gflags components.
Failed to find gflags - Could not find gflags include directory, set GFLAGS_INCLUDE_DIR to directory containing gflags/gflags.h
Failed to find glog - Could not find glog include directory, set GLOG_INCLUDE_DIR to directory containing glog/logging.h
Module opencv_sfm disabled because the following dependencies are not found: Eigen Glog/Gflags
Tesseract:   NO
Processing WORLD modules...
    module opencv_cudev...
    module opencv_core...
Allocator metrics storage type: 'long long'
    module opencv_cudaarithm...
    module opencv_flann...
    module opencv_imgproc...
    module opencv_intensity_transform...
    module opencv_ml...
    module opencv_phase_unwrapping...
    module opencv_plot...
    module opencv_quality...
    module opencv_reg...
    module opencv_surface_matching...
    module opencv_cudafilters...
    module opencv_cudaimgproc...
    module opencv_cudawarping...
    module opencv_dnn...
Registering hook 'INIT_MODULE_SOURCES_opencv_dnn': Z:/IA CAI/opencv-4.6.0/modules/dnn/cmake/hooks/INIT_MODULE_SOURCES_opencv_dnn.cmake
    module opencv_dnn_superres...
    module opencv_features2d...
    module opencv_fuzzy...
    module opencv_hfs...
    module opencv_imgcodecs...
imgcodecs: OpenEXR codec is disabled in runtime. Details: https://github.com/opencv/opencv/issues/21326
    module opencv_line_descriptor...
    module opencv_photo...
    module opencv_saliency...
    module opencv_text...
    module opencv_videoio...
    module opencv_wechat_qrcode...
    module opencv_xphoto...
    module opencv_barcode...
    module opencv_calib3d...
    module opencv_cudacodec...
    module opencv_cudafeatures2d...
    module opencv_cudastereo...
    module opencv_datasets...
    module opencv_highgui...
highgui: using builtin backend: WIN32UI
    module opencv_mcc...
    module opencv_objdetect...
    module opencv_rapid...
    module opencv_rgbd...
rgbd: Eigen support is disabled. Eigen is Required for Posegraph optimization
    module opencv_shape...
    module opencv_structured_light...
    module opencv_video...
    module opencv_xfeatures2d...
    module opencv_ximgproc...
    module opencv_xobjdetect...
    module opencv_aruco...
    module opencv_bgsegm...
    module opencv_bioinspired...
    module opencv_ccalib...
    module opencv_cudabgsegm...
    module opencv_cudalegacy...
    module opencv_cudaobjdetect...
    module opencv_dnn_objdetect...
    module opencv_dpm...
    module opencv_face...
    module opencv_gapi...
    module opencv_optflow...
    module opencv_stitching...
    module opencv_tracking...
    module opencv_cudaoptflow...
Building with NVIDIA Optical Flow API 2.0
    module opencv_stereo...
    module opencv_superres...
    module opencv_videostab...
Processing WORLD modules... DONE
Excluding from source files list: <BUILD>/modules/world/layers/layers_common.rvv.cpp
Found 'misc' Python modules from Z:/IA CAI/opencv-4.6.0/modules/python/package/extra_modules
Found 'mat_wrapper;utils' Python modules from Z:/IA CAI/opencv-4.6.0/modules/core/misc/python/package
Found 'gapi' Python modules from Z:/IA CAI/opencv-4.6.0/modules/gapi/misc/python/package
Found 'misc' Python modules from Z:/IA CAI/opencv-4.6.0/modules/python/package/extra_modules
Found 'mat_wrapper;utils' Python modules from Z:/IA CAI/opencv-4.6.0/modules/core/misc/python/package
Found 'gapi' Python modules from Z:/IA CAI/opencv-4.6.0/modules/gapi/misc/python/package
CMake Warning at cmake/OpenCVGenSetupVars.cmake:54 (message):
  CONFIGURATION IS NOT SUPPORTED: validate setupvars script in install
  directory
Call Stack (most recent call first):
  CMakeLists.txt:1039 (include)

General configuration for OpenCV 4.6.0 =====================================
  Version control:               unknown

  Extra modules:
    Location (extra):            Z:/IA CAI/opencv_contrib-4.6.0/modules
    Version control (extra):     unknown

  Platform:
    Timestamp:                   2022-08-14T15:17:22Z
    Host:                        Windows 10.0.19043 AMD64
    CMake:                       3.24.0
    CMake generator:             Visual Studio 17 2022
    CMake build tool:            Z:/VS CODE/VS/MSBuild/Current/Bin/amd64/MSBuild.exe
    MSVC:                        1932
    Configuration:               Release

  CPU/HW features:
    Baseline:                    SSE SSE2 SSE3
      requested:                 SSE3
    Dispatched code generation:  SSE4_1 SSE4_2 FP16 AVX AVX2 AVX512_SKX
      requested:                 SSE4_1 SSE4_2 AVX FP16 AVX2 AVX512_SKX
      SSE4_1 (18 files):         + SSSE3 SSE4_1
      SSE4_2 (2 files):          + SSSE3 SSE4_1 POPCNT SSE4_2
      FP16 (1 files):            + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 AVX
      AVX (5 files):             + SSSE3 SSE4_1 POPCNT SSE4_2 AVX
      AVX2 (33 files):           + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 FMA3 AVX AVX2
      AVX512_SKX (8 files):      + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 FMA3 AVX AVX2 AVX_512F AVX512_COMMON AVX512_SKX

  C/C++:
    Built as dynamic libs?:      YES
    C++ standard:                11
    C++ Compiler:                Z:/VS CODE/VS/VC/Tools/MSVC/14.32.31326/bin/Hostx64/x64/cl.exe  (ver 19.32.31332.0)
    C++ flags (Release):         /DWIN32 /D_WINDOWS /W4 /GR  /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi  /fp:fast     /EHa /wd4127 /wd4251 /wd4324 /wd4275 /wd4512 /wd4589 /MP  /MD /O2 /Ob2 /DNDEBUG 
    C++ flags (Debug):           /DWIN32 /D_WINDOWS /W4 /GR  /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi  /fp:fast     /EHa /wd4127 /wd4251 /wd4324 /wd4275 /wd4512 /wd4589 /MP  /MDd /Zi /Ob0 /Od /RTC1 
    C Compiler:                  Z:/VS CODE/VS/VC/Tools/MSVC/14.32.31326/bin/Hostx64/x64/cl.exe
    C flags (Release):           /DWIN32 /D_WINDOWS /W3  /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi  /fp:fast     /MP   /MD /O2 /Ob2 /DNDEBUG 
    C flags (Debug):             /DWIN32 /D_WINDOWS /W3  /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi  /fp:fast     /MP /MDd /Zi /Ob0 /Od /RTC1 
    Linker flags (Release):      /machine:x64  /INCREMENTAL:NO 
    Linker flags (Debug):        /machine:x64  /debug /INCREMENTAL 
    ccache:                      NO
    Precompiled headers:         NO
    Extra dependencies:          cudart_static.lib nppc.lib nppial.lib nppicc.lib nppidei.lib nppif.lib nppig.lib nppim.lib nppist.lib nppisu.lib nppitc.lib npps.lib cublas.lib cudnn.lib cufft.lib -LIBPATH:Z:/IA CAI/Programfile/Dev/lib/x64 -LIBPATH:Z:/IA CAI/Programfile/Dev/lib
    3rdparty dependencies:

  OpenCV modules:
    To be built:                 aruco barcode bgsegm bioinspired calib3d ccalib core cudaarithm cudabgsegm cudacodec cudafeatures2d cudafilters cudaimgproc cudalegacy cudaobjdetect cudaoptflow cudastereo cudawarping cudev datasets dnn dnn_objdetect dnn_superres dpm face features2d flann fuzzy gapi hfs highgui img_hash imgcodecs imgproc intensity_transform line_descriptor mcc ml objdetect optflow phase_unwrapping photo plot python3 quality rapid reg rgbd saliency shape stereo stitching structured_light superres surface_matching text tracking ts video videoio videostab wechat_qrcode world xfeatures2d ximgproc xobjdetect xphoto
    Disabled:                    -
    Disabled by dependency:      -
    Unavailable:                 alphamat cvv freetype hdf java julia matlab ovis python2 python2 sfm viz
    Applications:                tests perf_tests apps
    Documentation:               NO
    Non-free algorithms:         NO

  Windows RT support:            NO

  GUI: 
    Win32 UI:                    YES
    VTK support:                 NO

  Media I/O: 
    ZLib:                        build (ver 1.2.12)
    JPEG:                        build-libjpeg-turbo (ver 2.1.2-62)
    WEBP:                        build (ver encoder: 0x020f)
    PNG:                         build (ver 1.6.37)
    TIFF:                        build (ver 42 - 4.2.0)
    JPEG 2000:                   build (ver 2.4.0)
    OpenEXR:                     build (ver 2.3.0)
    HDR:                         YES
    SUNRASTER:                   YES
    PXM:                         YES
    PFM:                         YES

  Video I/O:
    DC1394:                      NO
    FFMPEG:                      YES (prebuilt binaries)
      avcodec:                   YES (58.134.100)
      avformat:                  YES (58.76.100)
      avutil:                    YES (56.70.100)
      swscale:                   YES (5.9.100)
      avresample:                YES (4.0.0)
    GStreamer:                   NO
    DirectShow:                  YES
    Media Foundation:            YES
      DXVA:                      YES

  Parallel framework:            Concurrency

  Trace:                         YES (with Intel ITT)

  Other third-party libraries:
    Intel IPP:                   2020.0.0 Gold [2020.0.0]
           at:                   Z:/IA CAI/build_nova/3rdparty/ippicv/ippicv_win/icv
    Intel IPP IW:                sources (2020.0.0)
              at:                Z:/IA CAI/build_nova/3rdparty/ippicv/ippicv_win/iw
    Lapack:                      NO
    Eigen:                       NO
    Custom HAL:                  NO
    Protobuf:                    build (3.19.1)

  NVIDIA CUDA:                   YES (ver 11.7, CUFFT CUBLAS FAST_MATH)
    NVIDIA GPU arch:             50
    NVIDIA PTX archs:

  cuDNN:                         YES (ver 8.5.0)

  OpenCL:                        YES (NVD3D11)
    Include path:                Z:/IA CAI/opencv-4.6.0/3rdparty/include/opencl/1.2
    Link libraries:              Dynamic load

  Python 3:
    Interpreter:                 C:/Users/Azazel/anaconda3/python.exe (ver 3.9.12)
    Libraries:                   C:/Users/Azazel/anaconda3/libs/python39.lib (ver 3.9.12)
    numpy:                       C:/Users/Azazel/anaconda3/lib/site-packages/numpy/core/include (ver 1.23.2)
    install path:                C:/Users/Azazel/anaconda3/Lib/site-packages/cv2/python-3.9

  Python (for build):            C:/Users/Azazel/anaconda3/python.exe

  Java:                          
    ant:                         NO
    JNI:                         NO
    Java wrappers:               NO
    Java tests:                  NO

  Install to:                    Z:/IA CAI/install
-----------------------------------------------------------------

Configuring done
asmorkalov commented 1 year ago

"but the kernel restart when compiling..." - what does it mean? Do you mean build failure or run-time network inference issue?

IltonWhatever commented 1 year ago

in the execution of the code the kernel restarts

IltonWhatever commented 1 year ago

image

IltonWhatever commented 1 year ago

This happens when I use this in code: net.setPreferableBackend(cv2.dnn.DNN_BACKEND_CUDA) net.setPreferableTarget(cv2.dnn.DNN_TARGET_CUDA)

asmorkalov commented 1 year ago

Please provide complete code reproducer and the model.

IltonWhatever commented 1 year ago

pip install opencv-python < Install before running

LOAD DEPENDENCIES

import time import cv2

CLASS COLORS

CONFIDENCE_THRESHHOLD = 0.6 NMS_THRESHOLD = 0.6 COLORS = [(0, 255, 255), (255, 255, 0), (0, 255, 0), (255, 0, 0)] txt = [0,0,0] # Text color

Loading the classes

class_name = [] with open('coco.names','r') as f: class_names = [cname.strip() for cname in f.readlines()]

Loading NEURAL Net Weight > tiny = light

net = cv2.dnn.readNet('yolov4-tiny.weights', 'yolov4-tiny.cfg')

Defining the use of GPU in the application

net.setPreferableBackend(cv2.dnn.DNN_BACKEND_OPENCV)

net.setPreferableTarget(cv2.dnn.DNN_TARGET_OPENCL_FP16)

net.setPreferableBackend(cv2.dnn.DNN_BACKEND_CUDA) net.setPreferableTarget(cv2.dnn.DNN_TARGET_CUDA)

Parameters of the Neural Network

model = cv2.dnn_DetectionModel(net) model.setInputParams(size=(416, 416), scale=1/255)

Video Capture

#Use 0 to use webcam or put the path of a video between its extension''
    #Example: cap = cv2.VideoCapture('videoplayback.mp4')

cap = cv2.VideoCapture('videoplayback.mp4')

Reading the video frames

prev_frame = 0 # Previous frame, used to calculate FPS start = 0 while True:

# Frame capture
_, frame = cap.read()

# Window Size
#frame = cv2.resize(frame, (1080, 780))

# Start of MS count
start = time.time()

# detection
classes, scores, boxes = model.detect(frame, CONFIDENCE_THRESHHOLD, NMS_THRESHOLD)

# End of MS Count
end = time.time()

# Cycle through all detections
for (classid, score, box) in zip(classes, scores, boxes):

    # Generating colors for classes
    color = COLORS[int(classid) % len(COLORS)]
    txt = tuple(txt)

    # Getting the class name by ID and its Score
    label = f'{class_names[classid]} : {int(score*100)}%'

    # Drawing the detection box
    cv2.rectangle(frame, box, color, 1)

    # Writing the class name on top of the object box
    cv2.putText(frame, label, (box[0], box[1] - 10), cv2.FONT_HERSHEY_SIMPLEX, 0.5, txt, 3) # Colored
    cv2.putText(frame, label, (box[0], box[1] - 10), cv2.FONT_HERSHEY_SIMPLEX, 0.5, color, 1) # Black part

# Calculating the time it took to make the detection
    #FPS
fps_label = 1/(start-prev_frame)
prev_frame = start
fps_label = int(fps_label)
fps_label = str(fps_label)

# Writing the FPS on the Image
cv2.putText(frame, fps_label, (0, 25), cv2.FONT_HERSHEY_SIMPLEX, 1, (0, 0 , 0), 5)
cv2.putText(frame, fps_label, (0, 25), cv2.FONT_HERSHEY_SIMPLEX, 1, (0, 255 , 0), 3)

# Creating the window
cv2.imshow('detections', frame)

# Pressing Q on the keyboard at any time to close the program
if cv2.waitKey(1) & 0xFF == ord('q'):
    break

Release the camera and destroy all windows

cap.release() cv2.destroyAllWindows()

asmorkalov commented 1 year ago

Could you attach the model too or, if it's confidential, attach the same model with random (zero) weights?

IltonWhatever commented 1 year ago

Model is "Yolo" Yolo Model.zip

asmorkalov commented 1 year ago

Thanks for the model. The model is loaded by model diagnostic tool without issues. Most probably it's CUDNN/CUDA specific issue.

IltonWhatever commented 1 year ago

did the code work using CUDA ?

zihaomu commented 1 year ago

Hi @WanliZhong, please take a look.

WanliZhong commented 1 year ago

@IltonWhatever Hi, I think you can try to downgrade the VS or CUDA version due to compatibility issues. I get the table below from the link https://quasar.ugent.be/files/doc/cuda-msvc-compatibility.html image

IltonWhatever commented 1 year ago

@IltonWhateverOi, acho que você pode tentar fazer o downgrade da versão VS ou CUDA devido a problemas de compatibilidade. Eu recebo a tabela abaixo do link https://quasar.ugent.be/files/doc/cuda-msvc-compatibility.html imagem

Ok, I am going to try