search

flashinfer-ai / flashinfer

FlashInfer: Kernel Library for LLM Serving
https://flashinfer.ai
Apache License 2.0
768 stars 64 forks source link

refactor: refactor decode handler #294

Closed yzh119 closed 4 weeks ago

yzh119 commented 4 weeks ago

Change the use of an optional fixed_grid_size to padded_batch_size.