Aruco - SEHException on detectMarkers() call

[HolographicAcademy]

Hey,

I'm using OpenCV 3.4.1 built with the Aruco module to do some marker detection.

To do so, I'm using a HoloLens device to capture camera frames, then send the frame from the Unity engine to a custom C++ DLL that expose OpenCV methods.

Everything works well, until I add a call to aruco::detectMarkers(). The result is "SEHException: External component has thrown an exception."

The exception does not come with any message that I could retrieve, and the ErrorCode is not set.

EDIT: Doing frame.empty() returns true, even if the byte array is correct. Any idead how to debug this from UWP?

System information

• OpenCV => 3.4.1 (including opencv_contrib modules, built for UWP)
• Operating System / Platform => UWP (x86 Win10)
• Compiler => Unity 2017.4.1f1/Visual Studio 2017

Custom C++ Layer (Aruco.dll)

extern "C" __declspec(dllexport) void scanFrame(BYTE* pixels, int height, int width)
{
    Mat frame = Mat(height, width, CV_8UC4, pixels);
    Ptr<Dictionary> arucoMarkers = getPredefinedDictionary(DICT_4X4_50);
    vector<int> detectedArucoIds;
    vector<vector<Point2f>> detectedArucoCorners;
    detectMarkers(frame, arucoMarkers, detectedArucoCorners, detectedArucoIds);
}

[HolographicAcademy]

Update: SEHExceptionis because the exception is thrown from unmanaged (C++) to my managed env (C#). The exception seems to be thrown by detectMarkers() during an Assert, because frame.isEmpty() returns true.

Only hack I found is to use:

cv::Mat gray;
cv::cvtColor(frame, gray, CV_RGBA2GRAY);

Then, I pass gray instead of frame to the detectMarkers() function.

I did not expect the Mat frame to be considered empty. Is this a normal behaviour ?

[NormandErwan]

Make sure the BYTE* pixels are not released by the garbage collector. The way your creating your frame, the Mat only initializes a header but doesn't copy the content. Using cv::cvtColor makes a copy of your content on the unmanaged memory, so you're safe. But you better keep a reference to the content in the managed memory to avoid being released until the scanFrame is done.

[HolographicAcademy]

@NormandErwan

That's exactly what I'm doing.

On the managed part I use GCHandle.Alloc() on the BYTE* pixels. It basically tells the garbage collector not to collect this memory until I free the GCHandle (ref. https://msdn.microsoft.com/en-us/library/1246yz8f(v=vs.110).aspx).

Code sample:

var frameBufferHandle = GCHandle.Alloc(buffer, GCHandleType.Pinned);

try
{
     var detectedCount = DetectMarkers(frameBufferHandle.AddrOfPinnedObject(), height, width);
}
finally
{
    frameBufferHandle.Free();
}