How can we use this lib to quantize Falcon7b / 40b models? - Githubissues

IST-DASLab / gptq

Code for the ICLR 2023 paper "GPTQ: Accurate Post-training Quantization of Generative Pretrained Transformers".

https://arxiv.org/abs/2210.17323

Apache License 2.0

1.81k stars 145 forks source link

How can we use this lib to quantize Falcon7b / 40b models? #36

Open tarunmcom opened 1 year ago

tarunmcom commented 1 year ago

How can we use this lib to quantize Falcon7b / 40b models?