Added steps to benchmark using the included mmlu script

turboderp / exllamav2

A fast inference library for running LLMs locally on modern consumer-class GPUs

MIT License

3.18k stars 233 forks source link

Closed RodriMora closed 3 weeks ago

RodriMora commented 3 weeks ago

Added steps to benchmark models using the included mmlu.py script. Also requirements that are not installed by default using the requirement.txt file