I cannot find FlexAttention-like api.

flashinfer-ai / flashinfer

FlashInfer: Kernel Library for LLM Serving

https://flashinfer.ai

Apache License 2.0

1.44k stars 135 forks source link

I cannot find FlexAttention-like api. #612

Open BirdChristopher opened 5 hours ago

BirdChristopher commented 5 hours ago

Hi, I'm pretty interested in the flexattention-like api mentioned in October lmsys meetup. But I can't find any introduction related. Is there anything I miss in documentation?

yzh119 commented 4 hours ago

You can try https://github.com/flashinfer-ai/flashinfer/blob/main/tests/test_jit_example.py at this moment.

The APIs are subject to changes because this feature is still experimental.