search

EnoxSoftware / MagicLeapWithOpenCVForUnityExample

MagicLeap with OpenCVforUnity Example
20 stars 8 forks source link

MagicLeapFaceDetectionExample - Only Blank Screen on Magic Leap #10

Closed artiransaygin closed 4 years ago

artiransaygin commented 4 years ago

First of all, thanks for the great project. I have tried the face detection scene with my webcam successfully and it beautifully works in real time. However, when I tried deploying the project to Magic Leap, the only thing I see is a blank screen. No menu, no nothing. I only need real-time face detection, so that is the only scene I added to the "Scenes in Build".

My LuminOS version is 0.98.10, I am using Unity 2019.2.0f1 and the OpenCV for Unity version is 2.3.8. I checked other issues like "MagicLeapArUcoExample is just a blank screen" but I don't see a solution there. One additional parameter I am questioning is the Minimum API Level which is 4 right now. Is there something I need to do to run a specific scene on Magic Leap other than following the ReadMe file? I would really appreciate it if you could help me. I really need to get this working. Thanks.

EnoxSoftware commented 4 years ago

Thank you very much for reporting. Have you already tried the setup by following the steps below? https://github.com/EnoxSoftware/MagicLeapWithOpenCVForUnityExample/issues/8#issuecomment-597754567

artiransaygin commented 4 years ago

Yes, I did. The only difference I can see is I imported Magic Leap Unity Package 0.22.0 and my Minimum API Level is 4. What is the purpose of Minimum API Level? Might there be something I am missing while building the project?

EnoxSoftware commented 4 years ago

Some of the methods require ComputerVision privilege. https://developer.magicleap.com/learn/guides/sdk-api-compatibilities

Also, when you install a new app, please uninstall the old one.

artiransaygin commented 4 years ago

Thanks for the link. Apparently, there was a version mismatch. I was using LuminOS 0.98.10 and I believe it requires Unity 2019.3.x, Lumin SDK 0.24.x and API Level 8. I corrected all of the versions, adjusted your code so that it works with the new Magic Leap unity package and for the first time I was able to see the menu. Unfortunately, even though I saw all the play, stop, change camera, etc. buttons I was not able to click them and the blank screen was still there. I felt like the screen was blocking the buttons but I am not sure.

My question is what should I see if the app is actually working and ready to detect. By the way, when I am switching the Unity platform to Lumin I am following the steps here: https://developer.magicleap.com/en-us/learn/guides/unity-setup

I am simply replacing the main camera in the project with Magic Leap's main camera and changing the "Target Eye" to left since that's where the camera is. Sad that I still get nothing. I really need to get this running and I am quite stuck right now, so I would love to here anymore suggestions.

EnoxSoftware commented 4 years ago

It seems that the API has changed significantly in Lumin SDK 0.24. I'm in the process of fixing MagicLeapWithOpenCVForUnityExample. I am terribly sorry, but please wait for a while.

artiransaygin commented 4 years ago

I would love to give it another shot with the updated version. Thanks for your effort.

EnoxSoftware commented 4 years ago

Thank you for your patience. Please try the new version.

MagicLeapWithOpenCVForUnityExample_v1.0.6.zip

Environment MagicLeapOne Lumin OS 0.98.10 Lumin SDK 0.24.1 Min API Level 8 Unity 2019.3.10f1 (64-bit) OpenCV for Unity 2.3.8

Setup

  1. Setup New Unity Project https://developer.magicleap.com/ja-jp/learn/guides/get-started-developing-in-unity
  2. Import the OpenCVForUnity2.3.8.
  3. Import the MagicLeapWithOpenCVForUnityExample.unitypackage 1.0.6.
magicleapwithopencvforunity_setup
  1. Check CameraCapture and ComputerVision checkbox in Publishing Settings. ( Minimum API Level 8 )magicleapwithopencvforunity_manifest_settings
  2. Build and Deploy to MagicLeap.

ScreenShot magicleapimagecaptureexample magicleapfacedetectionexample

artiransaygin commented 4 years ago

Thanks for the update. I followed each and every step and when I built and deployed only the MagicLeapFaceDetectionEample scene, a blank raw image was still the only thing I can see. Then, I tried deploying both scenes, MagicLeapFaceDetectionExample and MagicLeapImageCaptureExample and my environment appears only when I press the bumper. Does detection start only after pressing the bumper?

Secondly, I got multiple errors of the same kind after the capture started. You can find the screenshot down below. ML_20200424_15 58 36

EnoxSoftware commented 4 years ago

Have you completed the following steps?

Does detection start only after pressing the bumper?

Yes.

artiransaygin commented 4 years ago

I realised that I changed the code line where the cascade is loaded. This correction was needed for the older version of your project but apparently not for the updated version. Using it as you provided solves the issue. Thanks a lot.

I have a final question just to be sure. I am supposed to see a blank screen before pressing the bumper, correct? And any ideas on how to increase the success of detection? I feel like faces of people standing farther away cannot be detected as accurately.

EnoxSoftware commented 4 years ago

It is recommended to adjust the parameters of the detectMultiScale () method.

http://stackoverflow.com/questions/20801015/recommended-values-for-opencv-detectmultiscale-parameters http://answers.opencv.org/question/10654/how-does-the-parameter-scalefactor-in-detectmultiscale-affect-face-detection/ http://workpiles.com/2015/04/opencv-detectmultiscale-scalefactor/

artiransaygin commented 4 years ago

Thanks a lot. That's exactly what I needed.