search

ziplab / SAQ

This is the official PyTorch implementation for "Sharpness-aware Quantization for Deep Neural Networks".
Apache License 2.0
39 stars 7 forks source link

How to train 4 bit mobilenent v2 to 72.0 (in the paper)? #3

Open talenz opened 1 year ago

talenz commented 1 year ago

Great job! In this paper, the top1 of 4bit mobilenetv2 is 72.0 and this is beyond SOTA. Is there a way that can reproduce this result?

liujingcs commented 1 year ago

Thanks for your interest. As mentioned in the supplementary material, we fine-tune the quantized model with additional learnable layer-wise offsets for activations and knowledge distillation following [1][2]. Reference: [1] Nonuniform-to-Uniform Quantization: Towards Accurate Quantization via Generalized Straight-Through Estimation. CVPR 2022. [2] PROFIT: A Novel Training Method for sub-4-bit MobileNet Models. ECCV 2020.

talenz commented 1 year ago

Will you provide the script to reproduce this result?