search

qwopqwop200 / GPTQ-for-LLaMa

4 bits quantization of LLaMA using GPTQ
Apache License 2.0
2.98k stars 457 forks source link

What is the right perplexity number? #263

Open JianbangZ opened 1 year ago

JianbangZ commented 1 year ago

for base FP16 model --eval gives 5.68 PPL on wikitext2 --benchmark 2048 gives 6.43 on wikitext2

What's the difference?