nbasyl / OFQ

The official implementation of the ICML 2023 paper OFQ-ViT
MIT License
27 stars 0 forks source link

The +0.5 term in StatQ results in asymmetric weight quantization. #4

Open LuletterSoul opened 9 months ago

LuletterSoul commented 9 months ago

Unlike conventional symmetric weight quantization, I found that StatQ will let rounded integer weight add a term (+ 0.5), This item may cause the StatQ not to be actually deployed on hardware.

image

If we remove the -0.5 and +0.5 terms, the integer value of the weight will have an extra value, that is, 2**(bit - 1) -1. How do you modify this formula and achieve the same performance?