search

yuhuixu1993 / qa-lora

Official PyTorch implementation of QA-LoRA
MIT License
112 stars 10 forks source link

merge err: scales and qzeros dimension mismatch #15

Open sscheng216 opened 9 months ago

sscheng216 commented 9 months ago

Dear Sir,

Thanks for your sharing of this great work. I followed the instruction in readme.md that using GPTQ-for-LLaMa to quantize the llama-2-7b-chat model and applied lora finetuning, but I got error when using merge.py due to the dimension mismatch of scales and qzeros: image

Did I miss something?

yuhuixu1993 commented 9 months ago

@sscheng216 hi， you may check the type of zeros. You need to transfer the zeros from coded format into fp16

StiphyJay commented 9 months ago

@sscheng216 hi， you may check the type of zeros. You need to transfer the zeros from coded format into fp16

Hi, how to transfer the zeros from coded format into fp16

xiangxiangGao1996 commented 8 months ago

hi, i also want to ask how to transfer the zeros from coded format into fp16

YuanzeSun commented 5 months ago

Hi, I also meet this problem? Anyone solved it? Thank you!