search

EnoxSoftware / OpenCVForUnity

OpenCV for Unity (Untiy Asset Plugin)
https://assetstore.unity.com/packages/tools/integration/opencv-for-unity-21088
550 stars 172 forks source link

How to use ARCamera (Vuforia) with OpenCVForUnity? #173

Open bumchikiboss opened 8 months ago

bumchikiboss commented 8 months ago

I want to implement Vuforia's ARCamera with TextRecognitionCRNNWebCamExample. It uses default main camera and script - WebCamTextureToMatHelper. It throws timeout error when using AR camera. What do I need to do in order to use AR Camera with Text Recognition. Sorry I'm quite novice in this.

EnoxSoftware commented 8 months ago

WebCamTextureToMatHelper cannot be used because WebCamTexture is not used in Vuforia's ARCamera. This repository is an example of how to convert from Vuforia's ARCamera to OpenCV's Mat. https://github.com/EnoxSoftware/VuforiaWithOpenCVForUnityExample/

bumchikiboss commented 8 months ago

Ok so I have used the PostRenderToMatExample.cs from the repo you mentioned and put elements from TextRecognitionCRNNWebCamExample script in it. Its working with ARCamera but the thing is that its very slow. the framerate I'm getting is like 5-6 fps. is there any solution for this...maybe I can tinker with some values or something.

EnoxSoftware commented 8 months ago

There is a way to do an asynchronous conversion from Texture to Mat using the Rendering.AsyncGPUReadback class. Asynchronous conversion may improve FPS. https://docs.unity3d.com/2021.2/Documentation/ScriptReference/Rendering.AsyncGPUReadback.html

The following code is an example of an asynchronous conversion using the Rendering.AsyncGPUReadback class in OnPostRender method. https://github.com/EnoxSoftware/OpenCVForUnity/blob/2.5.7/Assets/OpenCVForUnity/Examples/MainModules/videoio/VideoWriterAsyncExample/VideoWriterAsyncExample.cs#L188-L271