search

mit-han-lab / smoothquant

[ICML 2023] SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models
https://arxiv.org/abs/2211.10438
MIT License
1.2k stars 138 forks source link

How to calculate Alpha? #54

Open Triple-L opened 1 year ago

Triple-L commented 1 year ago

Hi, thanks for sharing this fancy paper Smoothquant. I just have a few simple questions about the parameter Alpha. Will appreciate it if you guys can provide more details about it.

  1. How to define the outliers? Is it for per-channel? or calculate among the whole activation tensor?
  2. How to get the ratio of outliers. As you mentioned in the paper, for example, 30% outliers, how do you get such a ratio?
  3. How to get the Alpha scale according to the ratio?

Thank you.