move dispatch for batch prefill

flashinfer-ai / flashinfer

FlashInfer: Kernel Library for LLM Serving

https://flashinfer.ai

Apache License 2.0

1.1k stars 98 forks source link

Closed abcdabcd987 closed 4 months ago

abcdabcd987 commented 4 months ago

move DISPATCH_PAGE_SIZE out of BatchPrefillWithPagedKVCacheWrapperDispatched