Higher than 8-bit Quantization not working properly!?

IntelLabs / distiller

Neural Network Distiller by Intel AI Lab: a Python package for neural network compression research. https://intellabs.github.io/distiller

Apache License 2.0

4.34k stars 799 forks source link

Higher than 8-bit Quantization not working properly!? #554

Open Amin-Azar opened 3 years ago

Amin-Azar commented 3 years ago

Thanks for this great framework! I was wondering if there is an explicit 'no' or a limitation for quantizing weights and/or activations to higher than 8 bits using asymmetric methods? When I tried 16/32 for weights, on asymetric_s (similarly for activations) the accuracy drops to 0.2% while it should improve.