search

IST-DASLab / gptq

Code for the ICLR 2023 paper "GPTQ: Accurate Post-training Quantization of Generative Pretrained Transformers".
https://arxiv.org/abs/2210.17323
Apache License 2.0
1.81k stars 145 forks source link

opt_eval error #9

Closed liangxiaoyun closed 1 year ago

liangxiaoyun commented 1 year ago

After quant opt-125m and save the quant model. When I use ‘opt_eval’, get an error： Only supports a single token currently