flashinfer-ai flashinfer issues

flashinfer-ai / flashinfer

FlashInfer: Kernel Library for LLM Serving

https://flashinfer.ai

Apache License 2.0

1.14k stars 102 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

Fix compile/assert on group_size

#247 Qubitium closed 3 months ago
1
Add group_size 7 and fix compat with Yi 1.5 34b

#246 Qubitium closed 4 months ago
3
multiple definition of `cuda::__3::pipeline...

#245 jpf888 opened 4 months ago
0
Move -Wno-switch-bool argument to cxx from nvcc

#244 mgerstgrasser closed 4 months ago
0
Compilation fails due to "-Wno-switch-bool" nvcc flag

#243 mgerstgrasser closed 4 months ago
0
能否支持Volta/Tesla架构？

#242 alexngng closed 1 month ago
2
bugfix: Fix dispatcher in src directory

#241 yzh119 closed 4 months ago
0
bugfix: fix the `generate_dispatch_inc` script

#240 yzh119 closed 4 months ago
0
compilation: Suppress switch bool warning

#239 yzh119 closed 4 months ago
0
sampling: expose sampling APIs in pytorch

#238 yzh119 closed 4 months ago
0
Support MLA (Multi-Head Latent Attention) in DeepSeek-v2

#237 yzh119 opened 4 months ago
4
doc: bump documentation version

#236 yzh119 closed 4 months ago
0
cmake: macro trimming

#235 yzh119 closed 4 months ago
0
ci: update release wheel yaml

#234 yzh119 closed 4 months ago
0
fix: remove 8 from default page size

#233 yzh119 closed 4 months ago
0
chore(main): release 0.0.5

#232 github-actions[bot] closed 2 months ago
1
fix: fix macro to suppress compilation warning

#231 yzh119 closed 4 months ago
0
Revert "ci: remove multi-threading in nvcc compile flags (#229)"

#230 yzh119 closed 4 months ago
0
ci: remove multi-threading in nvcc compile flags

#229 yzh119 closed 4 months ago
0
bugfix: fix MANIFEST.in

#228 yzh119 closed 4 months ago
0
Support torch 2.3

#227 rkooo567 closed 4 months ago
3
bugfix: fix the potential issue of sampling kernels

#226 yzh119 closed 4 months ago
0
bugfix: Fix the correctness issue of sampling kernel

#225 yzh119 closed 4 months ago
0
Fix implicit cast in sampling

#224 abcdabcd987 closed 4 months ago
0
support versatile gqa size for batch prefill

#223 xuzhenqi closed 4 months ago
3
TypeError: get_cu_file_str() missing 1 required positional argument: 'idtype'

#222 xuzhenqi closed 4 months ago
1
bugfix: fix sampler's implementation bug when dtype is not float32

#221 yzh119 closed 4 months ago
0
cmake: fix cmake files

#220 yzh119 closed 4 months ago
0
misc: make max_top_p/k_rounds a input argument instead of template parameter

#219 yzh119 closed 4 months ago
0
fix: revert #144

#218 yzh119 closed 4 months ago
1
[BugFix] Fix build error related to dispatch page size

#217 esmeetu closed 4 months ago
1
ci: add pytorch 2.3 to matrix

#216 yzh119 closed 4 months ago
0
[TVMWrapper] Add wrapper functions for sampler

#215 MasterJH5574 closed 4 months ago
0
misc: parallel sampling from probability

#214 yzh119 closed 4 months ago
0
sampling: support parallel top-p sampling

#213 yzh119 closed 4 months ago
0
perm: optimize sampling performance

#212 yzh119 closed 4 months ago
0
sampling: fix alignment issue for vocab_size not divisible by vec_size

#211 yzh119 closed 4 months ago
0
dependency: update submodules

#210 yzh119 closed 4 months ago
0
move dispatch for batch prefill

#209 abcdabcd987 closed 4 months ago
0
bench: add sampling & norm benchmarks

#208 yzh119 closed 4 months ago
0
misc: fused kernel for sampling and normalization functions

#207 yzh119 closed 4 months ago
0
[CMAKE] Make generation option configuration

#205 tqchen closed 5 months ago
0
Fixes a misformated macros

#204 sighingnow closed 5 months ago
0
Update CMakeLists.txt

#203 shreygupta2809 closed 5 months ago
0
Vllm support

#202 MikeChenfu closed 1 week ago
1
Enable GQA group size = 6

#201 vinx13 closed 5 months ago
1
feat:support any num_heads for get_alibi_slope

#200 yz-tang closed 5 months ago
0
[LoRA] Roadmap of LoRA operators

#199 yzh119 opened 5 months ago
1
[CMake] Add positional independent code (PIC) option to kernels

#198 MasterJH5574 closed 5 months ago
3
remove duplicates of _get_cache_buf func

#197 HSQ79815 closed 5 months ago
0

Previous Next