Got accuray=0 when trying _real_int8_demo.ipynb

mit-han-lab / smoothquant

[ICML 2023] SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models

https://arxiv.org/abs/2211.10438

MIT License

1.1k stars 127 forks source link

Got accuray=0 when trying _real_int8_demo.ipynb #68

Open leocnj opened 6 months ago

leocnj commented 6 months ago

Tried both OPT-1.3B and OPT-6.7B models but met a lengthy warning and accuray = 0 when using smoothquanted model. Details can be found at #62. Any clue on solving this issue.