The reshape of input_id doesn't match HF OPT model's API

IST-DASLab / gptq

Code for the ICLR 2023 paper "GPTQ: Accurate Post-training Quantization of Generative Pretrained Transformers".

https://arxiv.org/abs/2210.17323

Apache License 2.0

1.81k stars 145 forks source link

Closed brian-fb closed 1 year ago

brian-fb commented 1 year ago

The reshape of input_ids should preserve the batch dimension.

efrantar commented 1 year ago

Thanks for the fix!