search

OpenGVLab / OmniQuant

[ICLR2024 spotlight] OmniQuant is a simple and powerful quantization technique for LLMs.
MIT License
689 stars 53 forks source link

[fix] 'QuantLlamaDecoderLayer' object has no attribute 'model_attn' #48

Closed xingchensong closed 9 months ago

xingchensong commented 9 months ago

if let is enabled, the attribute name used in smooth_and_quant_inplace should be self_attn

image

ChenMnZ commented 9 months ago

Thanks for your time for this PR. This is a significant mistake due to my omission.