Memory issues with OpenCV and unreal

[yonasteodros]

We are working on Aruco marker detection with opencv like you did (by implementing the detection and tracking part inside a custom opencv contrib module). unfortunately we are facing continuous crashing when using our code within unreal. It seems like the problem is due to garbage collection and/or memory allocation not working correctly between unreal and opencv's dlls. we found your plugin and it is working. Could you please give us some information about how you handled this kind of issues ?

for example in your code you mention "crashing memory deallocator" :

https://github.com/adynathos/AugmentedUnreality/blob/4ffee2052c05bb6a11a02721db196f3d2e29d0c0/ThirdParty/opencv/src/opencv_contrib/modules/augmented_unreality/include/opencv2/augmented_unreality/FiducialPattern.hpp#L106

Thank you!

[adynathos]

Aruco's detectMarkers requires passing a std::vector< std::vector< cv::Point2f > > (like here). There was a crash when this vector was deleted (when it went out of scope) in code which was part of the UE project. I moved it to an OpenCV module (dynamically linked to the UE project) and now it is fine.

I can speculate about the mechanism:

• That std::vector was created in the UE module and passed by reference to OpenCV
• OpenCV was adding new elements, causing the vector to allocate memory. But the version of std::vector inside OpenCV (or the method of allocation) was different than the one used in UE.
• When UE got the vector back and wanted to delete it, deleting that memory allocated by OpenCV was wrong and caused the crash

[yonasteodros]

@adynathos Thanks! I used the shared pointer to handle the crush.

[dwyycc]

Hello, I also want to apply ArUco to Unreal, but I haven't found a good way to directly identify it through an ArUco mark, so that the floor of my real space and the floor of the virtual space are merged. Could you please share it~