search

megvii-research / FQ-ViT

[IJCAI 2022] FQ-ViT: Post-Training Quantization for Fully Quantized Vision Transformer
Apache License 2.0
301 stars 48 forks source link

Dequantization #6

Closed ebsrn closed 2 years ago

ebsrn commented 2 years ago

Hi, thank you for sharing the coding.

Could you please explain why you dequantize the values after quantization?

https://github.com/linyang-zhh/FQ-ViT/blob/16122ee7ea33e80aed3edd29cfebb3ab2ce2cb69/models/ptq/quantizer/base.py#L46

linyang-zhh commented 2 years ago

@ebsrn Hi! In order to ensure the flexibility of future work (e.g. supporting training or fine-tuning), we use FakeQuantization. More information can be found in this link and this paper.

ebsrn commented 2 years ago

Thank you.

ebsrn commented 2 years ago

Hi again, I'm trying to do the inference with quantized weights. For this, I deleted the following line. Is this a correct approach? When I do this, I see that the output of the layer before the classification layer consists entirely of nan values.

https://github.com/linyang-zhh/FQ-ViT/blob/16122ee7ea33e80aed3edd29cfebb3ab2ce2cb69/models/ptq/quantizer/base.py#L46

linyang-zhh commented 2 years ago

@ebsrn Hi, quant and dequant are the standard and necessary steps in quantization and you can't remove any of them. These steps are introduced in the above-mentioned documents, and Figure 6 of this paper. I hope you could read these articles carefully, in order to enhance the understanding of quantization.