IST-DASLab / gptq

Code for the ICLR 2023 paper "GPTQ: Accurate Post-training Quantization of Generative Pretrained Transformers".
https://arxiv.org/abs/2210.17323
Apache License 2.0
1.92k stars 153 forks source link

Minor fix for llama #16

Closed Xiuyu-Li closed 1 year ago

Xiuyu-Li commented 1 year ago

It seems that llama.py does not feed position_ids and will throw out an error when running it. This PR addresses this issue and can run the llama script and reproduce results successfully.

efrantar commented 1 year ago

Thanks for fix, I must have run the code with an older HF version.