search

IST-DASLab / gptq

Code for the ICLR 2023 paper "GPTQ: Accurate Post-training Quantization of Generative Pretrained Transformers".
https://arxiv.org/abs/2210.17323
Apache License 2.0
1.92k stars 153 forks source link

About `--sym` zero point #19

Closed tpoisonooo closed 1 year ago

tpoisonooo commented 1 year ago

If , self.zero=8 as implemented here.

I thought self.zero=0 according to quantization doc or some inference code.

Is there any standard or consensus about symmetric quantization ?

efrantar commented 1 year ago

We are here simulating quantization; self.zero = 8 means that quantized values are in range [-8, 7] which should correspond to standard symmetric quantization as in the doc you linked (which e.g. shows [-128, 127] for 8-bit). In reality, 0 is still 0 during calculations, the self.zero is only used for number representation.