opencv / opencv

Open Source Computer Vision Library
https://opencv.org
Apache License 2.0
75.95k stars 55.62k forks source link

Version 4.9 ,The inference results of the ONNX model differ between GPU inference and CPU inference. #25512

Closed 123467895 closed 1 week ago

123467895 commented 2 weeks ago

System Information

// example for c++ user OpenCV version: 4.9.0 Operating System / Platform: Ubuntu 20.04 /Windwos11 Compiler & compiler version: GCC 9.3.0

Detailed description

The model is yolov8n.pt from the official YOLOv8 website. It was exported using the command "yolo detect export model=yolov8n.pt format=onnx opset=12".

use cpu: set “this->net.setPreferableBackend(cv::dnn::DNN_BACKEND_OPENCV); this->net.setPreferableTarget(cv::dnn::DNN_TARGET_CPU);” The results are correct. Partial results screenshot: 图片 4 coordinates + 80 class probabilities

use CUDA: set “this->net.setPreferableBackend(cv::dnn::DNN_BACKEND_CUDA); this->net.setPreferableTarget(cv::dnn::DNN_TARGET_CUDA);” The results are error. Partial results screenshot: 图片 The probabilities are present, but all 4 coordinates are 0. The coordinates are missing.

image: 1(1)

model: yolov8n.zip

Issue submission checklist

LaurentBerger commented 2 weeks ago

No error update your opencv version

import numpy as np
import cv2 as cv

yolov8n = cv.dnn.readNet("yolov8n.onnx")
yolov8n.setPreferableBackend(cv.dnn.DNN_BACKEND_OPENCV)
yolov8n.setPreferableTarget(cv.dnn.DNN_TARGET_CPU)
img = cv.imread(cv.samples.findFile('squirrel_cls.jpg'))

paramYolov8n = cv.dnn.Image2BlobParams()
paramYolov8n.ddepth = cv.CV_32F
paramYolov8n.borderValue = 0
paramYolov8n.paddingmode = cv.dnn.DNN_PMODE_NULL
paramYolov8n.datalayout = cv.dnn.DNN_LAYOUT_NCHW
paramYolov8n.mean = (0, 0, 0)
paramYolov8n.scalefactor = (1. / 255, 1. / 255, 1. / 255)
paramYolov8n.size = (640, 640)
paramYolov8n.swapRB = True
blobIn = cv.dnn.blobFromImageWithParams(img, paramYolov8n)
yolov8n.setInput(blobIn)
out = yolov8n.forward()

yolov8n_cuda = cv.dnn.readNet("yolov8n.onnx")
yolov8n_cuda.setPreferableBackend(cv.dnn.DNN_BACKEND_CUDA)
yolov8n_cuda.setPreferableTarget(cv.dnn.DNN_TARGET_CUDA)
yolov8n_cuda.setInput(blobIn)
out_cuda = yolov8n_cuda.forward()
print("Quadratic error ", np.mean((out[0]-out_cuda[0])**2))
print("Max error ", np.max(np.abs(out[0]-out_cuda[0])))

Quadratic error 5.6130234e-10 Max error 0.0027160645

123467895 commented 2 weeks ago

No error update your opencv version

import numpy as np
import cv2 as cv

yolov8n = cv.dnn.readNet("yolov8n.onnx")
yolov8n.setPreferableBackend(cv.dnn.DNN_BACKEND_OPENCV)
yolov8n.setPreferableTarget(cv.dnn.DNN_TARGET_CPU)
img = cv.imread(cv.samples.findFile('squirrel_cls.jpg'))

paramYolov8n = cv.dnn.Image2BlobParams()
paramYolov8n.ddepth = cv.CV_32F
paramYolov8n.borderValue = 0
paramYolov8n.paddingmode = cv.dnn.DNN_PMODE_NULL
paramYolov8n.datalayout = cv.dnn.DNN_LAYOUT_NCHW
paramYolov8n.mean = (0, 0, 0)
paramYolov8n.scalefactor = (1. / 255, 1. / 255, 1. / 255)
paramYolov8n.size = (640, 640)
paramYolov8n.swapRB = True
blobIn = cv.dnn.blobFromImageWithParams(img, paramYolov8n)
yolov8n.setInput(blobIn)
out = yolov8n.forward()

yolov8n_cuda = cv.dnn.readNet("yolov8n.onnx")
yolov8n_cuda.setPreferableBackend(cv.dnn.DNN_BACKEND_CUDA)
yolov8n_cuda.setPreferableTarget(cv.dnn.DNN_TARGET_CUDA)
yolov8n_cuda.setInput(blobIn)
out_cuda = yolov8n_cuda.forward()
print("Quadratic error ", np.mean((out[0]-out_cuda[0])**2))
print("Max error ", np.max(np.abs(out[0]-out_cuda[0])))

Quadratic error 5.6130234e-10 Max error 0.0027160645 图片

The classification accuracy is not significantly different in the later part, but the coordinates of the first four columns are different, and the results are all 0. The results are abnormal in version 4.9, while the results are correct in versions 4.7 and 4.8.

LaurentBerger commented 2 weeks ago

May be you should clean your post. Please update to last opencv version and post result of getBuildInformation()

123467895 commented 2 weeks ago

May be you should clean your post. Please update to last opencv version and post result of getBuildInformation()

Okay, please take a look at my results.:

General configuration for OpenCV 4.9.0 ===================================== Version control: unknown

Extra modules: Location (extra): D:/opencv4.9/opencv_contrib-4.x/modules Version control (extra): unknown

Platform: Timestamp: 2024-04-29T06:17:30Z Host: Windows 10.0.22631 AMD64 CMake: 3.28.1 CMake generator: Visual Studio 17 2022 CMake build tool: C:/Program Files/Microsoft Visual Studio/2022/Community/MSBuild/Current/Bin/amd64/MSBuild.exe MSVC: 1937 Configuration: Debug Release

CPU/HW features: Baseline: SSE SSE2 SSE3 requested: SSE3 Dispatched code generation: SSE4_1 SSE4_2 FP16 AVX AVX2 AVX512_SKX requested: SSE4_1 SSE4_2 AVX FP16 AVX2 AVX512_SKX SSE4_1 (16 files): + SSSE3 SSE4_1 SSE4_2 (1 files): + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 (0 files): + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 AVX AVX (8 files): + SSSE3 SSE4_1 POPCNT SSE4_2 AVX AVX2 (36 files): + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 FMA3 AVX AVX2 AVX512_SKX (5 files): + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 FMA3 AVX AVX2 AVX_512F AVX512_COMMON AVX512_SKX

C/C++: Built as dynamic libs?: YES C++ standard: 11 C++ Compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.37.32822/bin/Hostx64/x64/cl.exe (ver 19.37.32825.0) C++ flags (Release): /DWIN32 /D_WINDOWS /W4 /GR /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi /fp:precise /EHa /wd4127 /wd4251 /wd4324 /wd4275 /wd4512 /wd4589 /wd4819 /MP -openmp /O2 /Ob2 /DNDEBUG C++ flags (Debug): /DWIN32 /D_WINDOWS /W4 /GR /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi /fp:precise /EHa /wd4127 /wd4251 /wd4324 /wd4275 /wd4512 /wd4589 /wd4819 /MP -openmp /Zi /Ob0 /Od /RTC1 C Compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.37.32822/bin/Hostx64/x64/cl.exe C flags (Release): /DWIN32 /D_WINDOWS /W3 /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi /fp:precise /MP -openmp /O2 /Ob2 /DNDEBUG C flags (Debug): /DWIN32 /D_WINDOWS /W3 /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi /fp:precise /MP -openmp /Zi /Ob0 /Od /RTC1 Linker flags (Release): /machine:x64 delayimp.lib /DELAYLOAD:nvcuda.dll /DELAYLOAD:nvml.dll /IGNORE:4199 /INCREMENTAL:NO Linker flags (Debug): /machine:x64 delayimp.lib /DELAYLOAD:nvcuda.dll /DELAYLOAD:nvml.dll /IGNORE:4199 /debug /INCREMENTAL ccache: NO Precompiled headers: NO Extra dependencies: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.2/lib/x64/cudart_static.lib C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.2/lib/x64/nppial.lib C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.2/lib/x64/nppc.lib C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.2/lib/x64/nppitc.lib C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.2/lib/x64/nppig.lib C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.2/lib/x64/nppist.lib C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.2/lib/x64/nppidei.lib C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.2/lib/x64/cublas.lib C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.2/lib/x64/cublasLt.lib C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.2/lib/x64/cufft.lib C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.2/lib/x64/nppif.lib C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.2/lib/x64/nppim.lib C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.2/lib/x64/nppicc.lib 3rdparty dependencies:

OpenCV modules: To be built: aruco bgsegm bioinspired calib3d ccalib core cudaarithm cudabgsegm cudacodec cudafeatures2d cudafilters cudaimgproc cudalegacy cudaobjdetect cudaoptflow cudastereo cudawarping cudev datasets dnn dnn_objdetect dnn_superres dpm face features2d flann fuzzy gapi hfs highgui img_hash imgcodecs imgproc intensity_transform line_descriptor mcc ml objdetect optflow phase_unwrapping photo plot python3 quality rapid reg rgbd saliency shape signal stereo stitching structured_light superres surface_matching text tracking ts video videoio videostab world ximgproc xobjdetect xphoto Disabled: python_tests wechat_qrcode xfeatures2d Disabled by dependency: - Unavailable: alphamat cannops cvv freetype hdf java julia matlab ovis python2 python2 sfm viz Applications: apps Documentation: NO Non-free algorithms: NO

Windows RT support: NO

GUI: Win32 UI: YES VTK support: NO

Media I/O: ZLib: build (ver 1.3) JPEG: build-libjpeg-turbo (ver 2.1.3-62) SIMD Support Request: YES SIMD Support: NO WEBP: build (ver encoder: 0x020f) PNG: build (ver 1.6.37) TIFF: build (ver 42 - 4.2.0) JPEG 2000: build (ver 2.5.0) OpenEXR: build (ver 2.3.0) HDR: YES SUNRASTER: YES PXM: YES PFM: YES

Video I/O: DC1394: NO FFMPEG: YES (prebuilt binaries) avcodec: YES (58.134.100) avformat: YES (58.76.100) avutil: YES (56.70.100) swscale: YES (5.9.100) avresample: YES (4.0.0) GStreamer: NO DirectShow: YES Media Foundation: YES DXVA: YES

Parallel framework: OpenMP

Trace: YES (with Intel ITT)

Other third-party libraries: Intel IPP: 2021.11.0 [2021.11.0] at: D:/opencv4.9/opencv-4.9.0/build/3rdparty/ippicv/ippicv_win/icv Intel IPP IW: sources (2021.11.0) at: D:/opencv4.9/opencv-4.9.0/build/3rdparty/ippicv/ippicv_win/iw Lapack: NO Eigen: NO Custom HAL: NO Protobuf: build (3.19.1) Flatbuffers: builtin/3rdparty (23.5.9)

NVIDIA CUDA: YES (ver 12.2.91, CUFFT CUBLAS FAST_MATH) NVIDIA GPU arch: 75 NVIDIA PTX archs: 75

cuDNN: YES (ver 8.9.6)

OpenCL: YES (NVD3D11) Include path: D:/opencv4.9/opencv-4.9.0/3rdparty/include/opencl/1.2 Link libraries: Dynamic load

Python 3: Interpreter: D:/aconda3/run/envs/opencv49/python.exe (ver 3.10.9) Libraries: D:/aconda3/run/libs/python310.lib (ver 3.10.9) numpy: D:/aconda3/run/envs/opencv49/Lib/site-packages/numpy/core/include (ver 1.23.5) install path: D:/aconda3/run/envs/opencv49/Lib/site-packages/cv2/python-3.10

Python (for build): D:/aconda3/run/envs/opencv49/python.exe

Java:
ant: NO Java: YES (ver 1.8.0.381) JNI: C:/Program Files/Java/jdk-1.8/include C:/Program Files/Java/jdk-1.8/include/win32 C:/Program Files/Java/jdk-1.8/include Java wrappers: NO Java tests: NO


LaurentBerger commented 2 weeks ago

Why version control unknown? Where did you get your opencv version? You should clone last opencv version and build it using cmake Since 4.9.0 version there is 413 commits

LaurentBerger commented 2 weeks ago

@EnoxSoftware Why do you reference this issue?

fengyuentau commented 1 week ago

Should have been fixed via https://github.com/opencv/opencv/pull/24834, which is targeted on 4.10 release. So 4.9 is not going to work. Clone the latest code and compile your own.