search

IST-DASLab / gptq

Code for the ICLR 2023 paper "GPTQ: Accurate Post-training Quantization of Generative Pretrained Transformers".
https://arxiv.org/abs/2210.17323
Apache License 2.0
1.95k stars 155 forks source link

ValueError: not enough values to unpack (expected 2, got 1) #18

Closed jinz2014 closed 1 year ago

jinz2014 commented 1 year ago

Hello, I tried your instruction and got a value error. Was I doing right for benchmarking ? Thank you.

CUDA_VISIBLE_DEVICES=0 python opt.py facebook/opt-125m c4 --wbits 3 --save opt125m-3bit.pt

CUDA_VISIBLE_DEVICES=0 python opt.py facebook/opt-125m c4 --load opt125m-3bit.pt --benchmark 128 Loading model ... Done. Found cached dataset json (/$HOME/.cache/huggingface/datasets/allenaijson/allenai--c4-6fbe877195f42de5/0.0.0/fe5dd6ea2639a6df622901539cb550cf8797e5a6b2dd7af1cf934bed8e233e6e) Found cached dataset json (/$HOME/.cache/huggingface/datasets/allenaijson/allenai--c4-efc3d4f4606f44bd/0.0.0/fe5dd6ea2639a6df622901539cb550cf8797e5a6b2dd7af1cf934bed8e233e6e) Benchmarking ... Traceback (most recent call last): File "/$HOME/gptq/opt.py", line 455, in ... File "/$HOM/mambaforge/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 637, in forward batch_size, seq_length = input_shape ValueError: not enough values to unpack (expected 2, got 1)

wejoncy commented 1 year ago

image Ran into the same issue, the change above can workaround it.

efrantar commented 1 year ago

I think this should now be resolved with 8eca10660e9e48e71333c1f79129a818e7926b70.