Demo code for Bloom model?

mit-han-lab / smoothquant

[ICML 2023] SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models

https://arxiv.org/abs/2211.10438

MIT License

1.2k stars 138 forks source link

Demo code for Bloom model? #63

Open llCurious opened 10 months ago

llCurious commented 10 months ago

Hi, @Guangxuan-Xiao . I try to test the Bloom model. You have provided the act_scales for Bloom models, could you provide the demo code for Bloom model as well?