EnoxSoftware / OpenCVForUnity

OpenCV for Unity (Untiy Asset Plugin)
https://assetstore.unity.com/packages/tools/integration/opencv-for-unity-21088
558 stars 175 forks source link

Capture.grab() isn't working for the Hololens 2 #132

Open mohGhazala96 opened 2 years ago

mohGhazala96 commented 2 years ago

We are trying to use Capture.grab() and Capture.retrieve on the Hololens 2. The reason for not using the WebCamTextureHelper is that we want to change the exposure for the Hololen's webcam. Using the "capture.set(Videoio.CAP_PROP_EXPOSURE, exposure);" We couldnt find a way of doing that using WebCamTextureHelper or any of the helper classes.

Is there is a way to resolve this issue? Or maybe is there another way of changing the exposure of the Hololen's webcam?

P.s. This code works fine on the desktop app. When we uncomment the "capture.set(Videoio.CAP_PROP_EXPOSURE, exposure);", The expsoure is also changed and the app functions right. It's only when we deploy it to the Hololens2.

Here is the Code `

       int requestedDeviceId = 0;
        capture = new VideoCapture();
        capture.open((int)requestedDeviceId);

        if (!capture.isOpened())
        {
            Debug.LogError("capture.isOpened() is false. " + "DeviceId:" + (int)requestedDeviceId);

            capture.release();

            return;
        }
        int requestedFPS = 60;

        // capture.set(Videoio.CAP_PROP_FRAME_WIDTH, requestedWidth);
        // capture.set(Videoio.CAP_PROP_FRAME_HEIGHT, requestedHeight);
        // capture.set(Videoio.CAP_PROP_FPS, (int)requestedFPS);
        // capture.set(Videoio.CAP_PROP_AUTO_EXPOSURE, 1);// manual mode
        // capture.set(Videoio.CAP_PROP_EXPOSURE, exposure);

        inputMat = new Mat();
        capture.grab() ;
        capture.retrieve(inputMat);

`

Here is the error we are getting real time

MicrosoftTeams-image

EnoxSoftware commented 2 years ago

Unfortunately, the camera input via OpenCV's VideoCapture class does not seem to support the UWP platform. As documentation states: https://docs.opencv.org/4.x/d0/da7/videoio_overview.html Each backend supports devices properties (cv::VideoCaptureProperties) in a different way or might not support any property at all.

I found a repository that contains a short and simple code to use the Locatable camera on Hololens2, but I couldn't find if it can change the exposure of the camera. https://github.com/cookieofcode/LabAssistVision And the following repository appears to contain code to use the camera with Hololens and change the exposure, white balance and ISO. https://github.com/microsoft/MRLightingTools-Unity https://github.com/microsoft/MRLightingTools-Unity/blob/master/Assets/MixedRealityToolkit.LightingTools/CameraCapture/Util/VideoDeviceControllerWrapperUWP.cs

In addition, I found a forum thread that might give some tips on changing the exposure of the hololens camera. https://forum.unity.com/threads/control-over-hololens-camera-exposure.441251/