search

shimat / opencvsharp

OpenCV wrapper for .NET
Apache License 2.0
5.41k stars 1.15k forks source link

Can't read two cameras from one USB line at the same time by OpenCVSharp, while it works fine with opencv-python. #846

Closed bnuzhouwei closed 4 years ago

bnuzhouwei commented 4 years ago

I have two cameras with RGB and IR from a same USB line. The camera names in device manage is:

The code is like:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using OpenCvSharp;
namespace Camera2
{
    class Program
    {
        static void Main(string[] args)
        {
            var cap0 = VideoCapture.FromCamera(0);
            var cap1 = VideoCapture.FromCamera(1);
            while (true)
            {
                var mat0 = cap0.RetrieveMat();
                var mat1 = cap1.RetrieveMat();
                Console.WriteLine($"{mat0.Size()},{mat1.Size()}"); // mat1 is always size 0.
            }
        }
    }
}

The RetrieveMat always retrieve a frame with 0 size in NET48.

But It works fine with opencv-python. Python 3.8 with opencv 4.1.230.

import cv2
capture = cv2.VideoCapture(0)
capture.set(3,640)
capture.set(4,480)
capture1 = cv2.VideoCapture(1)
capture1.set(3,640)
capture1.set(4,480)
while(True):
    ret, frame = capture.read()
    ret1, frame1 = capture1.read()
    cv2.imshow('frame', frame)
    cv2.imshow('frame1', frame1)
    if cv2.waitKey(1) == ord('q'):
        break

Does the two cameras have the same name, or share a USB line, or other reasons caused this issue?

blaisexen commented 4 years ago

Hi, I also encountered like this, I made modifications of DirectX Camera functions here, https://sourceforge.net/projects/csharp-web-cam-list/

but when I used it using OpenCvSharp 4.2.0 (23 Jan., 2020) it doesn't work anymore.

hoqamep commented 4 years ago

I think you should call Grab() method before RetrieveMat(), or call Read() method instead of RetrieveMat(). Read() method combines Grab() and Retrieve().

var cap = VideoCapture.FromCamera(0);
cap.Grab();
var mat = cap.RetrieveMat(mat);

or

var cap = VideoCapture.FromCamera(0);
var mat = new Mat();
cap.Read(mat);
bnuzhouwei commented 4 years ago

Call Read() method instead of RetrieveMat() can't solve the problem.

mat1 is still always size 0.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using OpenCvSharp;
namespace Camera2
{
    class Program
    {
        static void Main(string[] args)
        {
            var cap0 = VideoCapture.FromCamera(0);
            var cap1 = VideoCapture.FromCamera(1);
            var mat0 = new Mat();
            var mat1 = new Mat();
            while (true)
            {
                cap0.Read(mat0);
                cap1.Read(mat1);
                Console.WriteLine($"{mat0.Width},{mat1.Width}");
            }
        }
    }
}
shimat commented 4 years ago

What is the value of cap1.IsOpened() ?

bnuzhouwei commented 4 years ago

The value of the second tried open capture.IsOpened() is false. The first one is true. 

var cap0 = VideoCapture.FromCamera(0);
var cap1 = VideoCapture.FromCamera(1);
cap0.IsOpened() //true
cap1.IsOpened() //false
var cap1= VideoCapture.FromCamera(1);
var cap0 = VideoCapture.FromCamera(0);
cap0.IsOpened() //false
cap1.IsOpened() //true
shimat commented 4 years ago

It may be due to differences in the OpenCV build configuration between OpenCvSharp and your opencv-python.

// C#
Console.WriteLine(Cv2.GetBuildInformation());
# Python
print(cv2.getBuildInformation())
bnuzhouwei commented 4 years ago

Console.WriteLine(Cv2.GetBuildInformation());


General configuration for OpenCV 4.1.0 =====================================
  Version control:               unknown

  Extra modules:
    Location (extra):            D:/opencv/opencv_contrib/modules
    Version control (extra):     4.1.0

  Platform:
    Timestamp:                   2019-04-15T10:22:02Z
    Host:                        Windows 10.0.14393 AMD64
    CMake:                       3.14.0
    CMake generator:             Visual Studio 15 2017
    CMake build tool:            C:/Program Files (x86)/Microsoft Visual Studio/2017/Professional/MSBuild/15.0/Bin/MSBuild.exe
    MSVC:                        1916

  CPU/HW features:
    Baseline:                    SSE SSE2
      requested:                 SSE2
    Dispatched code generation:  SSE4_1 SSE4_2 FP16 AVX
      requested:                 SSE4_1 SSE4_2 AVX FP16
      SSE4_1 (13 files):         + SSE3 SSSE3 SSE4_1
      SSE4_2 (1 files):          + SSE3 SSSE3 SSE4_1 POPCNT SSE4_2
      FP16 (0 files):            + SSE3 SSSE3 SSE4_1 POPCNT SSE4_2 FP16 AVX
      AVX (4 files):             + SSE3 SSSE3 SSE4_1 POPCNT SSE4_2 AVX

  C/C++:
    Built as dynamic libs?:      NO
    C++ Compiler:                C:/Program Files (x86)/Microsoft Visual Studio/2017/Professional/VC/Tools/MSVC/14.16.27023/bin/Hostx86/x86/cl.exe  (ver 19.16.27030.1)
    C++ flags (Release):         /DWIN32 /D_WINDOWS /W4 /GR  /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi /fp:fast   /arch:SSE /arch:SSE2 /EHa /wd4127 /wd4251 /wd4324 /wd4275 /wd4512 /wd4589 /MP8   /MT /O2 /Ob2 /DNDEBUG
    C++ flags (Debug):           /DWIN32 /D_WINDOWS /W4 /GR  /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi /fp:fast   /arch:SSE /arch:SSE2 /EHa /wd4127 /wd4251 /wd4324 /wd4275 /wd4512 /wd4589 /MP8   /MTd /Zi /Ob0 /Od /RTC1
    C Compiler:                  C:/Program Files (x86)/Microsoft Visual Studio/2017/Professional/VC/Tools/MSVC/14.16.27023/bin/Hostx86/x86/cl.exe
    C flags (Release):           /DWIN32 /D_WINDOWS /W3  /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi /fp:fast   /arch:SSE /arch:SSE2   /MP8    /MT /O2 /Ob2 /DNDEBUG
    C flags (Debug):             /DWIN32 /D_WINDOWS /W3  /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi /fp:fast   /arch:SSE /arch:SSE2   /MP8  /MTd /Zi /Ob0 /Od /RTC1
    Linker flags (Release):      /machine:X86  /NODEFAULTLIB:atlthunk.lib /INCREMENTAL:NO  /NODEFAULTLIB:libcmtd.lib /NODEFAULTLIB:libcpmtd.lib /NODEFAULTLIB:msvcrtd.lib
    Linker flags (Debug):        /machine:X86  /NODEFAULTLIB:atlthunk.lib /debug /INCREMENTAL  /NODEFAULTLIB:libcmt.lib /NODEFAULTLIB:libcpmt.lib /NODEFAULTLIB:msvcrt.lib
    ccache:                      NO
    Precompiled headers:         YES
    Extra dependencies:          ade comctl32 gdi32 ole32 setupapi ws2_32 C:/vcpkg/installed/x86-windows-static/lib/tesseract305.lib C:/vcpkg/installed/x86-windows-static/lib/leptonica-1.76.0.lib
    3rdparty dependencies:       ittnotify libprotobuf zlib libjpeg-turbo libwebp libpng libtiff libjasper IlmImf quirc ippiw ippicv

  OpenCV modules:
    To be built:                 aruco bgsegm bioinspired calib3d ccalib core datasets dnn dnn_objdetect dpm face features2d flann fuzzy gapi hfs highgui img_hash imgcodecs imgproc line_descriptor ml objdetect optflow phase_unwrapping photo plot quality reg rgbd saliency shape stereo stitching structured_light superres surface_matching text tracking video videoio videostab xfeatures2d ximgproc xobjdetect xphoto
    Disabled:                    java_bindings_generator python_bindings_generator world
    Disabled by dependency:      -
    Unavailable:                 cnn_3dobj cudaarithm cudabgsegm cudacodec cudafeatures2d cudafilters cudaimgproc cudalegacy cudaobjdetect cudaoptflow cudastereo cudawarping cudev cvv freetype hdf java js matlab ovis python2 python3 sfm ts viz
    Applications:                -
    Documentation:               NO
    Non-free algorithms:         YES

  Windows RT support:            NO

  GUI:
    Win32 UI:                    YES
    VTK support:                 NO

  Media I/O:
    ZLib:                        build (ver 1.2.11)
    JPEG:                        build-libjpeg-turbo (ver 2.0.2-62)
    WEBP:                        build (ver encoder: 0x020e)
    PNG:                         build (ver 1.6.36)
    TIFF:                        build (ver 42 - 4.0.10)
    JPEG 2000:                   build (ver 1.900.1)
    OpenEXR:                     build (ver 1.7.1)
    HDR:                         YES
    SUNRASTER:                   YES
    PXM:                         YES
    PFM:                         YES

  Video I/O:
    DC1394:                      NO
    FFMPEG:                      YES (prebuilt binaries)
      avcodec:                   YES (58.35.100)
      avformat:                  YES (58.20.100)
      avutil:                    YES (56.22.100)
      swscale:                   YES (5.3.100)
      avresample:                YES (4.0.0)
    GStreamer:                   NO
    DirectShow:                  YES

  Parallel framework:            Concurrency

  Trace:                         YES (with Intel ITT)

  Other third-party libraries:
    Intel IPP:                   2019.0.0 Gold [2019.0.0]
           at:                   D:/opencv/4.1.0/mybuild/410-x86-v141-static/3rdparty/ippicv/ippicv_win/icv
    Intel IPP IW:                sources (2019.0.0)
              at:                D:/opencv/4.1.0/mybuild/410-x86-v141-static/3rdparty/ippicv/ippicv_win/iw
    Lapack:                      NO
    Eigen:                       NO
    Custom HAL:                  NO
    Protobuf:                    build (3.5.1)

  OpenCL:                        YES (NVD3D11)
    Include path:                D:/opencv/4.1.0/sources/3rdparty/include/opencl/1.2
    Link libraries:              Dynamic load

  Python (for build):            NO

  Java:
    ant:                         NO
    JNI:                         NO
    Java wrappers:               NO
    Java tests:                  NO

  Install to:                    D:/opencv/4.1.0/mybuild/410-x86-v141-static/install

print(cv2.getBuildInformation())

General configuration for OpenCV 4.1.2 =====================================
  Version control:               4.1.2

  Platform:
    Timestamp:                   2019-11-22T02:16:13Z
    Host:                        Windows 6.3.9600 AMD64
    CMake:                       3.15.5
    CMake generator:             Visual Studio 14 2015 Win64
    CMake build tool:            C:/Program Files (x86)/MSBuild/14.0/bin/MSBuild.exe
    MSVC:                        1900

  CPU/HW features:
    Baseline:                    SSE SSE2 SSE3
      requested:                 SSE3
    Dispatched code generation:  SSE4_1 SSE4_2 FP16 AVX AVX2
      requested:                 SSE4_1 SSE4_2 AVX FP16 AVX2 AVX512_SKX
      SSE4_1 (14 files):         + SSSE3 SSE4_1
      SSE4_2 (1 files):          + SSSE3 SSE4_1 POPCNT SSE4_2
      FP16 (0 files):            + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 AVX
      AVX (4 files):             + SSSE3 SSE4_1 POPCNT SSE4_2 AVX
      AVX2 (27 files):           + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 FMA3 AVX AVX2

  C/C++:
    Built as dynamic libs?:      NO
    C++ Compiler:                C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/x86_amd64/cl.exe  (ver 19.0.24241.7)
    C++ flags (Release):         /DWIN32 /D_WINDOWS /W4 /GR  /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi  /fp:precise     /EHa /wd4127 /wd4251 /wd4324 /wd4275 /wd4512 /wd4589 /MP2   /MT /O2 /Ob2 /DNDEBUG
    C++ flags (Debug):           /DWIN32 /D_WINDOWS /W4 /GR  /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi  /fp:precise     /EHa /wd4127 /wd4251 /wd4324 /wd4275 /wd4512 /wd4589 /MP2   /MTd /Zi /Ob0 /Od /RTC1
    C Compiler:                  C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/x86_amd64/cl.exe
    C flags (Release):           /DWIN32 /D_WINDOWS /W3  /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi  /fp:precise       /MP2    /MT /O2 /Ob2 /DNDEBUG
    C flags (Debug):             /DWIN32 /D_WINDOWS /W3  /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi  /fp:precise       /MP2  /MTd /Zi /Ob0 /Od /RTC1
    Linker flags (Release):      /machine:x64  /NODEFAULTLIB:atlthunk.lib /INCREMENTAL:NO  /NODEFAULTLIB:libcmtd.lib /NODEFAULTLIB:libcpmtd.lib /NODEFAULTLIB:msvcrtd.lib
    Linker flags (Debug):        /machine:x64  /NODEFAULTLIB:atlthunk.lib /debug /INCREMENTAL  /NODEFAULTLIB:libcmt.lib /NODEFAULTLIB:libcpmt.lib /NODEFAULTLIB:msvcrt.lib
    ccache:                      NO
    Precompiled headers:         YES
    Extra dependencies:          ade comctl32 gdi32 ole32 setupapi ws2_32
    3rdparty dependencies:       ittnotify libprotobuf zlib libjpeg-turbo libwebp libpng libtiff libjasper IlmImf quirc ippiw ippicv

  OpenCV modules:
    To be built:                 calib3d core dnn features2d flann gapi highgui imgcodecs imgproc ml objdetect photo python3 stitching video videoio
    Disabled:                    world
    Disabled by dependency:      -
    Unavailable:                 java js python2 ts
    Applications:                -
    Documentation:               NO
    Non-free algorithms:         NO

  Windows RT support:            NO

  GUI:
    Win32 UI:                    YES
    VTK support:                 NO

  Media I/O:
    ZLib:                        build (ver 1.2.11)
    JPEG:                        build-libjpeg-turbo (ver 2.0.2-62)
    WEBP:                        build (ver encoder: 0x020e)
    PNG:                         build (ver 1.6.37)
    TIFF:                        build (ver 42 - 4.0.10)
    JPEG 2000:                   build (ver 1.900.1)
    OpenEXR:                     build (ver 2.3.0)
    HDR:                         YES
    SUNRASTER:                   YES
    PXM:                         YES
    PFM:                         YES

  Video I/O:
    DC1394:                      NO
    FFMPEG:                      YES (prebuilt binaries)
      avcodec:                   YES (58.54.100)
      avformat:                  YES (58.29.100)
      avutil:                    YES (56.31.100)
      swscale:                   YES (5.5.100)
      avresample:                YES (4.0.0)
    GStreamer:                   NO
    DirectShow:                  YES
    Media Foundation:            YES
      DXVA:                      NO

  Parallel framework:            Concurrency

  Trace:                         YES (with Intel ITT)

  Other third-party libraries:
    Intel IPP:                   2019.0.0 Gold [2019.0.0]
           at:                   C:/projects/opencv-python/_skbuild/win-amd64-3.8/cmake-build/3rdparty/ippicv/ippicv_win/icv
    Intel IPP IW:                sources (2019.0.0)
              at:                C:/projects/opencv-python/_skbuild/win-amd64-3.8/cmake-build/3rdparty/ippicv/ippicv_win/iw
    Lapack:                      NO
    Eigen:                       NO
    Custom HAL:                  NO
    Protobuf:                    build (3.5.1)

  OpenCL:                        YES (NVD3D11)
    Include path:                C:/projects/opencv-python/opencv/3rdparty/include/opencl/1.2
    Link libraries:              Dynamic load

  Python 3:
    Interpreter:                 C:/Python38-x64/python.exe (ver 3.8)
    Libraries:                   optimized C:/Python38-x64/libs/python38.lib debug C:/Python38-x64/libs/python38_d.lib (ver 3.8.0)
    numpy:                       C:/Python38-x64/lib/site-packages/numpy/core/include (ver 1.17.3)
    install path:                python

  Python (for build):            C:/Python27-x64/python.exe

  Java:
    ant:                         NO
    JNI:                         C:/Program Files/Java/jdk1.8.0/include C:/Program Files/Java/jdk1.8.0/include/win32 C:/Program Files/Java/jdk1.8.0/include
    Java wrappers:               NO
    Java tests:                  NO

  Install to:                    C:/projects/opencv-python/_skbuild/win-amd64-3.8/cmake-install

Any methods to solve the problem?

shimat commented 4 years ago 
 Media Foundation:            YES

In cmake process of OpenCV for OpenCvSharp, -DWITH_MSMF=OFF was set. Perhaps your second camera device can not be captured without MSMF.

The reason MSMF is disabled in OpenCvSharp is to make it easy to work on Windows Server without MSMF.

bnuzhouwei commented 4 years ago

I can't fix it by replace opencv_ffmpeg410 by opencv_videoio_ffmpeg410.dll download from opencv site, should i rebuild opencvsharp?

bnuzhouwei commented 4 years ago

Which config file should i modify to enable Media Foundation? And when i build opencvsharpextern, a error LNK1104 can't open “opencv_world400d.lib” OpenCvSharpExtern C:\Users\python\Downloads\opencvsharp-4.1.0.20190417\src\OpenCvSharpExtern\LINK 1
Thanks.

bnuzhouwei commented 4 years ago

I could open the two camera by Aforge.Video,DirectShow, How can I modify opencvsharp to fix the issue, thanks

bnuzhouwei commented 4 years ago

I think it is none business of the MSMF, because both the camera can be open, but only the first called can be open.

shimat commented 4 years ago

According to your Cv2.GetBuildInformation, you appear to be using OpenCvSharp 4.1.0. Have you tried 4.2.0? 4.2.0 supports apiPreference parameter. 4.1.0 may mishandle the parameters of the native cv::VideoCapture.

https://github.com/shimat/opencvsharp/commit/d98b3d3d9e7ae2da63508ab76ad09bd62018bd29#diff-917de9f42824804e7f244c8d7258152eR61

bnuzhouwei commented 4 years ago

When I try OpenCvSharp 4.1.0 above, An error --- Unable to load DLL 'OpenCvSharpExtern' in 4.2.0 was throw may for AVX not support Intel J1900.Issues 818

stale[bot] commented 4 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.