search

EnoxSoftware / OpenCVForUnity

OpenCV for Unity (Untiy Asset Plugin)
https://assetstore.unity.com/packages/tools/integration/opencv-for-unity-21088
557 stars 175 forks source link

copyToMat crashes Unity Editor #147

Open AR-Vsx opened 2 years ago

AR-Vsx commented 2 years ago

Hi, I'm trying to decode a 16-bit png encoded depth image from the Kinect Azure. Since the depth image is transmitted via network I receive the data as a byte array. For some reason the Editor crashes after a while during the decoding process. Sometimes it even crashes right on startup.

Stacktrace:

=================================================================
    Managed Stacktrace:
=================================================================
      at <unknown> <0xffffffff>
      at OpenCVForUnity.UtilsModule.MatUtils:OpenCVForUnity_ByteArrayToMatData <0x0012c>
      at OpenCVForUnity.UtilsModule.MatUtils:copyToMat <0x0033a>
      at <OnEncoderBufferReceived>d__7:MoveNext <0x00b8a>
      at System.Runtime.CompilerServices.AsyncVoidMethodBuilder:Start <0x001f1>
      at SocketIOHandler:OnEncoderBufferReceived <0x0024a>
      at <>c__DisplayClass7_1:<OnUnityThread>b__1 <0x0007b>
      at UnityThread:Update <0x003cf>
      at System.Object:runtime_invoke_void__this__ <0x00187>
=================================================================

Obtained 29 stack frames
0x00007ff9912c1170 (opencvforunity) OpenCVForUnity_ByteArrayToMatData
0x000001ec1eb0012d (Mono JIT Code) (wrapper managed-to-native) OpenCVForUnity.UtilsModule.MatUtils:OpenCVForUnity_ByteArrayToMatData (intptr,intptr)
0x000001ec1eaff78b (Mono JIT Code) OpenCVForUnity.UtilsModule.MatUtils:copyToMat<byte> (byte[],OpenCVForUnity.CoreModule.Mat)
0x000001ec1e8f0b2b (Mono JIT Code) [SocketIOHandler.cs:81] SocketIOHandler/<OnEncoderBufferReceived>d__6:MoveNext () 
0x000001eb8b223d82 (Mono JIT Code) System.Runtime.CompilerServices.AsyncVoidMethodBuilder:Start<TStateMachine_REF> (TStateMachine_REF&)
0x000001ec1e8ed82b (Mono JIT Code) SocketIOHandler:OnEncoderBufferReceived (SocketIOClient.SocketIOResponse)
0x000001ec1e8ed59c (Mono JIT Code) [SocketIOUnity.cs:60] SocketIOUnity/<>c__DisplayClass7_1:<OnUnityThread>b__1 () 
0x000001eb8d5cc120 (Mono JIT Code) [UnityThread.cs:126] UnityThread:Update () 
0x000001eb8d3ad6d8 (Mono JIT Code) (wrapper runtime-invoke) object:runtime_invoke_void__this__ (object,intptr,intptr,intptr)
0x00007ff9a335e034 (mono-2.0-bdwgc) [mini-runtime.c:3445] mono_jit_runtime_invoke 
0x00007ff9a329e724 (mono-2.0-bdwgc) [object.c:3064] do_runtime_invoke 
0x00007ff9a329e8bc (mono-2.0-bdwgc) [object.c:3111] mono_runtime_invoke 
0x00007ff672055b04 (Unity) scripting_method_invoke
0x00007ff672050724 (Unity) ScriptingInvocation::Invoke
0x00007ff67201e0e4 (Unity) MonoBehaviour::CallMethodIfAvailable
0x00007ff67201e1ec (Unity) MonoBehaviour::CallUpdateMethod
0x00007ff671b1e268 (Unity) BaseBehaviourManager::CommonUpdate<BehaviourManager>
0x00007ff671b2584a (Unity) BehaviourManager::Update
0x00007ff671d3454d (Unity) `InitPlayerLoopCallbacks'::`2'::UpdateScriptRunBehaviourUpdateRegistrator::Forward
0x00007ff671d1b56c (Unity) ExecutePlayerLoop
0x00007ff671d1b643 (Unity) ExecutePlayerLoop
0x00007ff671d21259 (Unity) PlayerLoop
0x00007ff672c512a8 (Unity) PlayerLoopController::UpdateScene
0x00007ff672c4f48f (Unity) Application::TickTimer
0x00007ff67308cf2a (Unity) MainMessageLoop
0x00007ff6730917fb (Unity) WinMain
0x00007ff6743a398e (Unity) __scrt_common_main_seh
0x00007ffa61b27034 (KERNEL32) BaseThreadInitThunk
0x00007ffa62042651 (ntdll) RtlUserThreadStart

Code :

//depthBuffer contains the encoded png data as byte[]
Mat buff = new Mat(1, depthBuffer.Length, CvType.CV_16UC1);
MatUtils.copyToMat<byte>(depthBuffer, buff);

Mat decodedMat = Imgcodecs.imdecode(buff, Imgcodecs.IMREAD_ANYDEPTH);
buff.Dispose();
byte[] decodedBuff = new byte[depthBuffer.Length];
MatUtils.copyFromMat<byte>(decodedMat, decodedBuff);
decodedBuff.Dispose();

I tried disposing the Mats since it feels like a memory issue but with no luck. I'm using Unity 2021.2.10 and OpenCVForUnity 2.8.4 on Windows 10.

EnoxSoftware commented 2 years ago

Could you try changing it from CvType.CV_16UC1 to CvType.CV_8UC1?

//depthBuffer contains the encoded png data as byte[]
Mat buff = new Mat(1, depthBuffer.Length, CvType.CV_8UC1);
AR-Vsx commented 2 years ago

Changing the type worked, but as soon as I choose a higher png compression level than zero the Editor crashes instantly. I'm also using opencv to encode the depth image I receive from the Kinect in a seperate c++ application.