CUDA_VISIBLE_DEVICES=0 python llama.py /mnt/g/models/conceptofmind_LLongMA-2-13b c4 --wbits 4 --true-sequential --act-order --groupsize 32 --save_safetensors /mnt/g/models/LLongMA-2-13b-16k-GPTQ/4bit-32g-tsao.safetensors
Found cached dataset json (/home/anon/.cache/huggingface/datasets/allenai___json/allenai--c4-6fbe877195f42de5/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51)
Found cached dataset json (/home/anon/.cache/huggingface/datasets/allenai___json/allenai--c4-efc3d4f4606f44bd/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51)
Token indices sequence length is longer than the specified maximum sequence length for this model (8924 > 8192). Running this sequence through the model will result in indexing errors
Starting ...
Ready.
Quantizing layer 1/40..
+------------------+--------------+------------+-----------+-------+
| name | weight_error | fp_inp_SNR | q_inp_SNR | time |
+==================+==============+============+===========+=======+
Traceback (most recent call last):
File "/home/anon/GPTQ-for-LLaMa/llama.py", line 492, in <module>
quantizers = llama_sequential(model, dataloader, DEV)
File "/home/anon/miniconda3/envs/gptq/lib/python3.9/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/home/anon/GPTQ-for-LLaMa/llama.py", line 114, in llama_sequential
scale, zero, g_idx, error = gptq[name].fasterquant(percdamp=args.percdamp, groupsize=args.groupsize, actorder=args.act_order, name=name)
File "/home/anon/GPTQ-for-LLaMa/gptq.py", line 221, in fasterquant
self.print_loss(name=name, q_weight=Q, weight_error=error, timecost=(time.time() - tick))
File "/home/anon/GPTQ-for-LLaMa/gptq.py", line 110, in print_loss
if self.inp1 is not None:
AttributeError: 'GPTQ' object has no attribute 'inp1'
Not sure exactly what commit broke it, but I rolled back to an arbitrary commit from about 2 weeks ago and it works again
Not sure exactly what commit broke it, but I rolled back to an arbitrary commit from about 2 weeks ago and it works again