flashinfer-ai / flashinfer

FlashInfer: Kernel Library for LLM Serving
https://flashinfer.ai
Apache License 2.0
760 stars 64 forks source link

benchmark: add batch prefill with ragged kv-cache benchmark #338

Closed yzh119 closed 1 week ago