EnoxSoftware / MagicLeapWithOpenCVForUnityExample

MagicLeap with OpenCVforUnity Example
20 stars 8 forks source link

Face detection app pauses after around 2 minutes #11

Closed artiransaygin closed 4 years ago

artiransaygin commented 4 years ago

As titled, the face detection example pauses after 2 minutes. I am able to resume the capture by pressing the bumber but needing to do so after every 2 minutes is annoying. I thought of the possibility of a memory leak and tried disposing variables like "faces" or "grayMat", however doing so just causes a crash. What might be the reason behind these pauses? If it's a memory leak, what should I do to solve the problem? Thanks.

EnoxSoftware commented 4 years ago

Thank you very much for reporting.

The MagicLeapFaceDetectionExample is based on the VideoCaptureExample. Does VideoCaptureExample with the following line commented out work in rawVideoCaptureMode for more than 2 minutes? https://github.com/magicleap/MagicLeapUnitySDK/blob/master/Assets/MagicLeap/Examples/Scripts/VideoCaptureExample.cs#L136-L143

artiransaygin commented 4 years ago

It is already commented out and unfortunately, it pauses after 2 minutes. I tried including those commented out parts and increased the max recording time to 100 minutes but it did not change anything. I think memory overloads after around 2 minuted but disposing variable like "yuvMat" or "rgbMat" crashes the app without even starting.

I checked methods like "OnDestroy", "EndCapture", "DisableMLCamera", "OnDisable", etc. and none of them are being entered before the pause. In addition, I checked the bool variable "_isCapturing", "_isCameraConnected", "_hasStarted" and "_appPaused" to see if they are chenged to trigger a pause but they are not. "_appPaused" is false even though the capture stops. In addition, capture resumes when I press the bumper again.

EnoxSoftware commented 4 years ago

Does VideoCaptureExample's rawVideoCaptureMode work for more than 2 minutes?

artiransaygin commented 4 years ago

Yes, it definitely runs for more than 2 minutes and actually looks like it can go on forever.

Update: I increased the value of "scaleFactor" parameter in "detectMultiScale" to decrease the workload caused by detection and now the application runs for at least 10 minutes which is a big upgrade. However, now that the quality of detection decreased due to the increase in "scaleFactor", sometimes I get multiple bounding boxes for a single face and I want to select the one with the higher confidence. I checked the code but couldn't find a way to obtain bounding box confidence values. Is there way to get them?

EnoxSoftware commented 4 years ago

It is recommended to adjust the minNeighbors parameter. https://stackoverflow.com/questions/22249579/opencv-detectmultiscale-minneighbors-parameter

For confidence levels, it seems you can get rejectLevels and levelWeights. https://enoxsoftware.github.io/OpenCVForUnity/3.0.0/doc/html/class_open_c_v_for_unity_1_1_objdetect_module_1_1_cascade_classifier.html#ab492876a1b8dfcb4986acac71c24ba97

artiransaygin commented 4 years ago

I fixed the issue by slightly decreasing the scale factor so that it applies a more thorough detection.