search

IST-DASLab / gptq

Code for the ICLR 2023 paper "GPTQ: Accurate Post-training Quantization of Generative Pretrained Transformers".
https://arxiv.org/abs/2210.17323
Apache License 2.0
1.81k stars 145 forks source link

Is there a beginners guide to the GPTQ algorithm? #32

Closed vgoklani closed 1 year ago

vgoklani commented 1 year ago

I found the paper hard to follow, is there a beginner / dummys guide? Perhaps slides from a talk or tutorial? Thanks!

efrantar commented 1 year ago

First, I would recommend to also take a look at our predecessor work OBC (which should explain some of the general ideas behind GPTQ) as well as our follow-up on sparsification SparseGPT (a related algorithm that can also be combined with GPTQ). Additionally, there is this talk on YouTube by one of us which covers both GPTQ and SparseGPT.

There also seems to be this pretty detailed video on GPTQ specifically; though it is not associated with us.

I hope some of these pointers are helpful!