qwopqwop200 / GPTQ-for-LLaMa

4 bits quantization of LLaMA using GPTQ
Apache License 2.0
2.98k stars 457 forks source link

Finetuning Quantized LLaMA #259

Open Qifeng-Wu99 opened 1 year ago

Qifeng-Wu99 commented 1 year ago

Hello,

I really appreciate your work done here.

I wonder if you could also release a python script on finetuning quantized LLaMA on a customized dataset.

It is inevitable that quantization would damage performance, while finetuning could make the model perform better on a user-desired dataset.

Thank you.