search

ARM-software / ComputeLibrary

The Compute Library is a set of computer vision and machine learning functions optimised for both Arm CPUs and GPUs using SIMD technologies.
2.87k stars 783 forks source link

Add per-channel quantization support for NEConvolutionLayer #955

Closed terfendail closed 2 years ago

terfendail commented 2 years ago

Problem description: Are there any plans to support QSYMM8_PER_CHANNEL datatype for NEConvolutionLayer input tensor? Are there any plans to support asymmetric per-channel quantization(something like QASYMM8_PER_CHANNEL/QASYMM8_SIGNED_PER_CHANNEL)?

morgolock commented 2 years ago

Hi @terfendail

What's your use case? Does your model require support for this?

ACL already supports QSYMM8_PER_CHANNEL for the weights: https://github.com/ARM-software/ComputeLibrary/blob/main/arm_compute/runtime/NEON/functions/NEConvolutionLayer.h#L106

Hope this helps.

terfendail commented 2 years ago

There are some models that could be quantized more conveniently using asymmetric per-channel quantization. However the case could be handled using existing implementation of course.

@alvoron discussion could be of interest for you