Fix memory inefficiency for baseline (GPTQ) configuration

Vahe1994 / SpQR

Apache License 2.0

515 stars 40 forks source link

Fix memory inefficiency for baseline (GPTQ) configuration #5

Closed justheuristic closed 1 year ago

justheuristic commented 1 year ago

This PR resolves an issue found by @poedator : the GPTQ baseline with very large group size would previously try to find outliers - and waste memory doing so - even though the outlier threshold is infinite

Vahe1994 commented 1 year ago

lgtm