Does not support 3bit quantization?

qwopqwop200 / GPTQ-for-LLaMa

4 bits quantization of LLaMA using GPTQ

Apache License 2.0

2.98k stars 457 forks source link

Open foamliu opened 1 year ago

foamliu commented 1 year ago

3-bit quant of a 65B model, encoutered following error during pack stage：