search

ARM-software / armnn

Arm NN ML Software. The code here is a read-only mirror of https://review.mlplatform.org/admin/repos/ml/armnn
https://developer.arm.com/products/processors/machine-learning/arm-nn
MIT License
1.14k stars 307 forks source link

ARMNN prepare API takes lot of time when using GPU #654

Closed abhajaswal closed 2 years ago

abhajaswal commented 2 years ago

Dear team,

I try to run mobilenetv2 model using ARMNN , but the prepare API for GPU target take 10 times more time than the CPU target

Tiime taken by ARMNN CPU to prepare : 992ms ARMNN GPU : 9607ms

Tiime taken by opensource tflite CPU plugin : 44ms

Could you let me know what could be the root cause? Does the pipline creation takes more time?

I need to review the usage of the ARMNN further for our Use case , but in case prepare takes lot of time then i need an understanding about it.

MikeJKelly commented 2 years ago

Hi @abhajaswal

The preparation time for CPU and GPU includes time taken to compile the kernels used by those backends. This can be quite time consuming but only occurs once.

Yesterday @morgolock gave some more information on the issue you raised on ComputeLibrary: https://github.com/ARM-software/ComputeLibrary/issues/977

Best regards, Mike

abhajaswal commented 2 years ago

Thanks Mike , I will check this. You can close the issue from here I think , as the improvements would be seen on ARMNN automatically once ACL side optimizations are done