GPTQ pseudo-quantization saved weights (pt format) How load Re-evaluation

IST-DASLab / gptq

Code for the ICLR 2023 paper "GPTQ: Accurate Post-training Quantization of Generative Pretrained Transformers".

https://arxiv.org/abs/2210.17323

Apache License 2.0

1.81k stars 145 forks source link

GPTQ pseudo-quantization saved weights (pt format) How load Re-evaluation #50

Open CXiaorong opened 7 months ago

CXiaorong commented 7 months ago

GPTQ pseudo-quantization saved weights (pt format) How load re-evaluation, I set the --load parameter after the execution, found an error: gptq-main\quant.py", line 636, in forward raise ValueError('Only supports a single token currently.')