search

OpenGVLab / OmniQuant

[ICLR2024 spotlight] OmniQuant is a simple and powerful quantization technique for LLMs.
MIT License
689 stars 53 forks source link

Update omniquant.py #42

Closed brisker closed 9 months ago

brisker commented 9 months ago

it seems that Smoothquant init scales with absmax, not max

ChenMnZ commented 9 months ago

Thanks for your proposal. However, the abs operation has been done in https://github.com/OpenGVLab/OmniQuant/blob/bd6001d959889ddd221c6f38afc5463a1edb3805/generate_act_scale_shift.py#L32

brisker commented 9 months ago

@ChenMnZ the code you mentioned is activation absmax，but what I mentioned is weight absmax

brisker commented 9 months ago

@ChenMnZ in original smoothquant, scales are calculated based on both act and weights , and both of them are absmax, not max https://github.com/mit-han-lab/smoothquant/blob/main/smoothquant/smooth.py#L19 image

ChenMnZ commented 9 months ago

You are right. Thanks for your correction.