search

qwopqwop200 / GPTQ-for-LLaMa

4 bits quantization of LLaMA using GPTQ
Apache License 2.0
2.99k stars 459 forks source link

LoRa and diff with bitsandbytes #281

Open RonanKMcGovern opened 1 year ago

RonanKMcGovern commented 1 year ago
  1. What changes would I need to make for GPTQ to support LoRa for Llama 2?
  2. What's the main difference between GPTQ vs bitsandbytes? Is it that GPTQ re-adjusts the weights to keep the same loss function shape?