search

megvii-research / FQ-ViT

[IJCAI 2022] FQ-ViT: Post-Training Quantization for Fully Quantized Vision Transformer
Apache License 2.0
301 stars 48 forks source link

QIntLayerNorm 讲 self.mode = 'int' #36

Closed PeiyanFlying closed 1 year ago

PeiyanFlying commented 1 year ago

您好，请问我将 QIntLayerNorm 中的 self.mode 设置成了 ‘int’。但是，程序报错，请问可能是哪里出现的问题呢？麻烦了！谢谢您！

PeiyanFlying commented 1 year ago

image

linyang-zhh commented 1 year ago

Please keep QIntLayerNorm in ln mode during the calibration step. After the calibration, call model.model_quant() and it will set QIntLayerNorm to int mode.

PeiyanFlying commented 1 year ago

Ok, sure! That works! Sorry about another question. I read the code and the function 'def get_MN(self, x)' in QIntLayerNorm. I am a little confused. What does the M and N represent? Does the N represent the 'a' in the Equation (8)? And the function 'def get_MN(self, x)' is the Equantion (11) ? image

image

I appreciate your help. Thanks so much!

linyang-zhh commented 1 year ago

No. They are in the Eq. (31) - (32) on the appendix of the paper.

PeiyanFlying commented 1 year ago

Ok, sure. Many thanks for your help!