search

ARM-software / ComputeLibrary

The Compute Library is a set of computer vision and machine learning functions optimised for both Arm CPUs and GPUs using SIMD technologies.
MIT License
2.83k stars 775 forks source link

gemm+silu fused operator is not supported in ACL #1125

Open TianyuLi0 opened 3 months ago

TianyuLi0 commented 3 months ago

Output of 'strings libarm_compute.so | grep arm_compute_version': arm_compute_version=v24.04 Build options: {'Werror': '1', 'build_dir': '//acl/build', 'debug': '0', 'neon': '1', 'opencl': '0', 'os': 'linux', 'openmp': '1', 'cppthreads': '0', 'arch': 'armv8.2-a', 'multi_isa': '1', 'fixed_format_kernels': '1', 'build': 'native'} Git hash=b'4fda7a803eaadf00ba36bd532481a33c18952089'

Platform: Neoverse N2

Operating System: Ubuntu 22.04

Problem description: gemm+silu fused operator is not supported in ACL

LLM use case: llama like models use silu activation with gemm.

Similar issue https://github.com/ARM-software/ComputeLibrary/issues/1083

morgolock commented 1 week ago

Hi @TianyuLi0

We currently don't have plans to implement silu in ACL.

KleidiAI is better suited to accelerate the type of workloads you find in these models than ACL. Please have a look at this article discussing LLM acceleration on mobile CPUs.

Alternatively you are welcome to submit a patch contributing to ACL. Please see our contribution guide for more information on how to add a new operator

Hope this helps