search

NVIDIA / kvpress

LLM KV cache compression made easy
Apache License 2.0
232 stars 9 forks source link

Add Infinitebench benchmark #11

Open maxjeblick opened 4 days ago

maxjeblick commented 4 days ago

🚀 Feature

Run benchmark on (a subset of tasks) for https://github.com/NVIDIA/kvpress/tree/main/evaluation/infinite_bench similar to loogle/ruler.