search

yhhhli / BRECQ

Pytorch implementation of BRECQ, ICLR 2021
MIT License
247 stars 56 forks source link

Quantization seems to be doesn't produce good accuracy ? Are there additional settings I missed? #40

Open padeirocarlos opened 8 months ago

padeirocarlos commented 8 months ago

So I tried running your code on my dataset with a pre-trained ResNet50 model. I got these results

Full precision model i got accuracy of : MobileNetV2 (58.19) Quantized model (W8A8) i got accuracy of : MobileNetV2 (12.02) Quantized model (W6A6) i got accuracy of : MobileNetV2 (10.12)

Full precision model i got accuracy of : ResNet-50 (65.16) Quantized model (W8A8) i got accuracy of : ResNet-50 (13.22) Quantized model (W6A6) i got accuracy of : ResNet-50 (11.02)

image https://github.com/yhhhli/BRECQ/blob/main/main_imagenet.py#L201C1-L229C87

My accuracy however does not come nearly as close to the float model which is around 58.19% and 65.16% but after quantization Are there additional settings I missed?

mj0219 commented 5 months ago

I encountered the same issue when using my pretrained model on CIFAR-10 for experiments. The accuracy is only about ten percent. Have you resolved this?

padeirocarlos commented 4 months ago

Hi, @mj0219 sorry, for later reply ! Could you share you model after quantization? The potentially issue can be related to batch-normal quantization process! Therefore, could share your model after quantization? I want to see batch-normal quantization result!