test: add DtypeKV template param in bench_batch_decode

flashinfer-ai / flashinfer

FlashInfer: Kernel Library for LLM Serving

https://flashinfer.ai

Apache License 2.0

1.48k stars 147 forks source link

Closed dc3671 closed 2 weeks ago

dc3671 commented 2 weeks ago

Add typename TKV so that it's convenient to benchmark FP8 KVCache. A example output: