Very low accuracy with torch vision mobilenet_v2 ptq model

intel / neural-compressor

SOTA low-bit LLM quantization (INT8/FP8/INT4/FP4/NF4) & sparsity; leading model compression techniques on TensorFlow, PyTorch, and ONNX Runtime

https://intel.github.io/neural-compressor/

Apache License 2.0

2.23k stars 257 forks source link

Very low accuracy with torch vision mobilenet_v2 ptq model #84

Closed soyebn closed 2 years ago

soyebn commented 2 years ago

I am using the following command at, .../neural-compressor/examples/pytorch/image_recognition/torchvision_models/quantization/ptq/cpu/ipex/ python main.py -t --ipex -a mobilenet_v2 --pretrained /data/ssd/datasets/ilsvrc/2012/data-folders/val/

But I am getting almost 0% accuracy with quantization. If I use -a resnet18, I get quantization accuracy close to float, so setup and other dataset related paths seems to be fine. Do I have to change anything else for mobilenet_v2?

chensuyue commented 2 years ago

Sorry for the late reply, ResNet18/Mobilenet_v2 should not support ipex yet, we will remove the example to avoid confusion. And the other 2 model(ResNet50 /ResNext101_32x16d ) in this page have been supported.