search

Unity-Technologies / barracuda-release

Other
565 stars 77 forks source link

Run Barracuda with ARCore and GPU acceleration #88

Closed scode-cmd closed 3 years ago

scode-cmd commented 4 years ago

Hi,

Is there any way to run Barracuda with Open GL on Android? CPU execution is slow and AR Core strictly requires Open GL as the graphics API. Thanks!

mantasp commented 4 years ago

Hi @fsal-code, we are working on some sort of solution for that, but initially it probably won't run all the models. It would be good to know for our prioritization process what architectures/models are you interested in?

scode-cmd commented 4 years ago

Hi @fsal-code, we are working on some sort of solution for that, but initially it probably won't run all the models. It would be good to know for our prioritization process what architectures/models are you interested in?

Thanks! We have our own computer vision models developed in Keres, they are feed forward and use Conv2D and Dense layers. We also have models for motion synthesis that use 1d convolution and operate on the temporal domain. We are interested in running both with AR Core.

raphipaffi commented 3 years ago

Hi @mantasp , we are in a similar situation. We need to run ARCore + Barracuda for Image Classification in our mobile app. Our model is based on MobileNetV2, so Conv2D with Relu, Conv1D with Relu, BatchNorm, MaxPool2D, AvgPool2D, GlobalAvgPool, Dense with Relu and Softmax would be great to have.

Here is a reference model for MobileNetV2 in ONNX format: https://github.com/onnx/models/blob/master/vision/classification/mobilenet/model/mobilenetv2-7.onnx

Thanks a lot for your support!

ROBYER1 commented 3 years ago

Yolo v3 and Yolo v4 models please, preferably from Darknet but I am aware of needing to convert those to ONNX first

AlexRibard commented 3 years ago

Side comment here: Barracuda does run on OpenGL on Android. Drivers are spotty however so we noticed some discrepancies and compilations errors due to driver support. So we cannot guarantee 100% valid execution, hence why we do not support it. But you can still make it run, so my advice would be to try and see. You might be lucky.

FlorentGuinier commented 3 years ago

We have test for Yolo-v3-tiny from darknet and Yolov4 (tiny_yolov4_mobilenetv2_lite) on Barracuda 1.4.0. However there are probably other variation of Yolov3 and v4 that do not import. Could you try the model you are targeting on Barracuda 1.4.0 please?

NicoLenssen commented 3 years ago

@FlorentGuinier : Is the tiny_yolov4_mobilenetv2_lite model that you are referring to available somewhere? And how did you convert it from darknet to ONNX?

FlorentGuinier commented 3 years ago

I think it came from https://github.com/hunglc007/tensorflow-yolov4-tflite but i'm actually not sure! Idea would be to convert from Darknet to TF using the above repo. Then from TF frozen graph to ONNX using https://docs.unity3d.com/Packages/com.unity.barracuda@2.0/manual/Exporting.html

FlorentGuinier commented 3 years ago

Closing as there is no activity on the thread, please reopen if needed.