search

flashinfer-ai / flashinfer

FlashInfer: Kernel Library for LLM Serving
https://flashinfer.ai
Apache License 2.0
822 stars 77 forks source link

sampling: expose sampling APIs in pytorch #238

Closed yzh119 closed 2 months ago

yzh119 commented 2 months ago

We only have C++ and TVM sampling APIs atm, this PR exposes sampling APIs in PyTorch.