search

SNU-ARC / any-precision-llm

[ICML 2024 Oral] Any-Precision LLM: Low-Cost Deployment of Multiple, Different-Sized LLMs
MIT License
83 stars 3 forks source link

Gemma-7B Perplexity Issue #1

Closed SyphonArch closed 9 months ago

SyphonArch commented 9 months ago

Quantization on Gemma-7B yields reasonable results in demo.py but results in absurdly high or infinite perplexity values on evaluation.

SyphonArch commented 9 months ago

Resolved by 1d6dfee Relevant issue