Activation scales for bloomz 7.1b

mit-han-lab / smoothquant

[ICML 2023] SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models

https://arxiv.org/abs/2211.10438

MIT License

1.1k stars 127 forks source link

Open bil-ash opened 7 months ago

bil-ash commented 7 months ago

The article at https://huggingface.co/blog/generative-ai-models-on-intel-cpu mentions that smoothquant was applied on bloomz 7b1 model also. But in https://huggingface.co/mit-han-lab/smoothquant-scales only smoothquant-scales for the 176b model is provided. Please add the smoothquant-scales for the 7b1 model also because many (like me) do not have the resources to run the 176b model

Arthur-Ling commented 6 months ago

you can usegenerate_act_scales.py to generate scales for bloom-7b1