search

qwopqwop200 / GPTQ-for-LLaMa

4 bits quantization of LLaMA using GPTQ
Apache License 2.0
2.99k stars 459 forks source link

explicitly declare wbits and group_size #212

Closed cauyxy closed 1 year ago

cauyxy commented 1 year ago

if you are using llama7b-4bit-128g.pt, you should explicitly declare wbits and group_size instead of using the default params