bench: add sampling & norm benchmarks

flashinfer-ai / flashinfer

FlashInfer: Kernel Library for LLM Serving

https://flashinfer.ai

Apache License 2.0

1.1k stars 98 forks source link

Closed yzh119 closed 4 months ago

yzh119 commented 4 months ago

Add benchmarking scripts for sampling & normalization C++ APIs. Fix a performance issues existed in rmsnorm implementation.