issues
search
flashinfer-ai
/
flashinfer
FlashInfer: Kernel Library for LLM Serving
https://flashinfer.ai
Apache License 2.0
822
stars
77
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
fix: fix macro to suppress compilation warning
#231
yzh119
closed
2 months ago
0
Revert "ci: remove multi-threading in nvcc compile flags (#229)"
#230
yzh119
closed
2 months ago
0
ci: remove multi-threading in nvcc compile flags
#229
yzh119
closed
2 months ago
0
bugfix: fix MANIFEST.in
#228
yzh119
closed
2 months ago
0
Support torch 2.3
#227
rkooo567
closed
2 months ago
3
bugfix: fix the potential issue of sampling kernels
#226
yzh119
closed
2 months ago
0
bugfix: Fix the correctness issue of sampling kernel
#225
yzh119
closed
2 months ago
0
Fix implicit cast in sampling
#224
abcdabcd987
closed
2 months ago
0
support versatile gqa size for batch prefill
#223
xuzhenqi
closed
2 months ago
3
TypeError: get_cu_file_str() missing 1 required positional argument: 'idtype'
#222
xuzhenqi
closed
2 months ago
1
bugfix: fix sampler's implementation bug when dtype is not float32
#221
yzh119
closed
2 months ago
0
cmake: fix cmake files
#220
yzh119
closed
2 months ago
0
misc: make max_top_p/k_rounds a input argument instead of template parameter
#219
yzh119
closed
2 months ago
0
fix: revert #144
#218
yzh119
closed
2 months ago
1
[BugFix] Fix build error related to dispatch page size
#217
esmeetu
closed
2 months ago
1
ci: add pytorch 2.3 to matrix
#216
yzh119
closed
2 months ago
0
[TVMWrapper] Add wrapper functions for sampler
#215
MasterJH5574
closed
2 months ago
0
misc: parallel sampling from probability
#214
yzh119
closed
2 months ago
0
sampling: support parallel top-p sampling
#213
yzh119
closed
2 months ago
0
perm: optimize sampling performance
#212
yzh119
closed
2 months ago
0
sampling: fix alignment issue for vocab_size not divisible by vec_size
#211
yzh119
closed
2 months ago
0
dependency: update submodules
#210
yzh119
closed
2 months ago
0
move dispatch for batch prefill
#209
abcdabcd987
closed
2 months ago
0
bench: add sampling & norm benchmarks
#208
yzh119
closed
2 months ago
0
misc: fused kernel for sampling and normalization functions
#207
yzh119
closed
2 months ago
0
[CMAKE] Make generation option configuration
#205
tqchen
closed
3 months ago
0
Fixes a misformated macros
#204
sighingnow
closed
3 months ago
0
Update CMakeLists.txt
#203
shreygupta2809
closed
3 months ago
0
Vllm support
#202
MikeChenfu
opened
3 months ago
0
Enable GQA group size = 6
#201
vinx13
closed
3 months ago
1
feat:support any num_heads for get_alibi_slope
#200
yz-tang
closed
3 months ago
0
[LoRA] Roadmap of LoRA operators
#199
yzh119
opened
3 months ago
1
[CMake] Add positional independent code (PIC) option to kernels
#198
MasterJH5574
closed
3 months ago
3
remove duplicates of _get_cache_buf func
#197
HSQ79815
closed
3 months ago
0
[Minor] Fix build when disable bf16 and fp8
#196
esmeetu
closed
3 months ago
0
[Install] Build error on main branch
#195
esmeetu
closed
2 months ago
0
Shared-prefix rope issue
#194
lkc1997
opened
3 months ago
1
[TVMWrapper] Support auxiliary DLTensor with byte offset
#193
MasterJH5574
closed
3 months ago
0
Compare Append Kernel's Results with Xformers
#192
LiuXiaoxuanPKU
closed
3 months ago
2
Does flashinfer support float datatype?
#191
ZSL98
opened
3 months ago
3
example: add example of using BatchPrefillWithRaggedKVCacheWrapper c++ api
#190
yzh119
closed
3 months ago
0
QUESTION: C++ API support Ragged Tensor now?
#189
yz-tang
closed
3 months ago
1
How was the data in the blog measured?
#188
cloudhan
opened
4 months ago
5
Make flashinfer kernels cuda graphs friendly
#187
AgrawalAmey
closed
4 weeks ago
12
falshinfer build error
#186
yz-tang
closed
4 months ago
1
perf: optimize warp layout for prefill operator for small query length
#185
yzh119
closed
1 month ago
3
[fix] change build error to runtime error to allow build pass with older architecture.
#184
guocuimi
closed
4 months ago
0
refactor: unify dispatch scheme
#183
yzh119
closed
4 months ago
0
fix: fix python package dispatch error message
#182
yzh119
closed
4 months ago
0
[BUG] model Yi-34B compat
#181
Qubitium
closed
1 month ago
1
Previous
Next