search

mlc-ai / mlc-llm

Universal LLM Deployment Engine with ML Compilation
https://llm.mlc.ai/
Apache License 2.0
19.26k stars 1.58k forks source link

[Fix] Disable FlashInfer when sliding window is enabled #3026

Closed MasterJH5574 closed 1 week ago

MasterJH5574 commented 1 week ago

This PR fixes the function table initialization, so that when sliding window is enabled, we won't pick the FlashInfer attn kernel.