search

ARM-software / ComputeLibrary

The Compute Library is a set of computer vision and machine learning functions optimised for both Arm CPUs and GPUs using SIMD technologies.
MIT License
2.75k stars 767 forks source link

Is Weights Manager supported? #1052

Closed alvoron closed 7 months ago

alvoron commented 1 year ago

Hello,

I've found Weights Manager feature described in https://arm-software.github.io/ComputeLibrary/v23.02.1/architecture.xhtml#architecture_weights_manager

I've found that Weights Manager is used in several functions (e.g. NEGEMM, NEFullyConnectedLayer) however, IWeightsManager::acquire and IWeightsManager::run methods are never called.

Tests against IWeightsManager class are also missed. Is this feature workable? Is there any example/test how to use it?

Thank you.

morgolock commented 8 months ago

Hi @alvoron

The Weights Manager is only used in Fully Connecter Layer and is going to be deprecated soon as it is no longer actively maintained.

IWeightsManager is the interface of an internal ACL component that let us reuse already transformed weights in multiple functions at runtime and in this way avoid having to reshape the weights many times. This improves the performance and reduces the memory usage.

If you are looking way to improve memory consumption in general I would suggest you look into TensorAllocator::import_memory(), see import_memory

You can also see how to import memory in our validation tests for import_memory here

We are going to improve the documentation for importing memory in the release 24.02

Hope this helps.