issues
search
WukLab
/
preble
Stateful LLM Serving
Apache License 2.0
32
stars
4
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
ModuleNotFoundError: No module named 'sglang.srt.managers.router'
#75
LiuZhetan
opened
1 week ago
1
The keys that do not exist in the response chunk dictionary
#74
ljjlovefree
opened
2 months ago
1
requests version conflict
#73
0x6b7966
opened
3 months ago
1
Feature/basic cli release
#72
vikranth22446
closed
5 months ago
0
Update global scheduler with better perf
#71
vikranth22446
closed
5 months ago
0
Refactor/rebase
#70
jiange91
closed
5 months ago
0
Add NSDI 2024 spring development updates
#69
vikranth22446
closed
5 months ago
0
Cleanup/cleanup post nsdi
#68
vikranth22446
closed
5 months ago
0
Ckpt/video qa
#67
jiange91
closed
5 months ago
0
Feat/partial eviction
#66
jiange91
closed
5 months ago
0
Add new dataset programing
#65
vikranth22446
closed
5 months ago
0
Feature/initial worst case benchmarks
#64
vikranth22446
closed
5 months ago
0
Feat/hc disagg
#63
jiange91
closed
5 months ago
0
Feat/hc disagg
#62
jiange91
closed
5 months ago
0
Add topt to scheduler
#61
vikranth22446
closed
5 months ago
0
Add mixed workloads
#60
dongmingli-Ben
closed
5 months ago
1
Add LOC support
#59
vikranth22446
closed
5 months ago
0
Add support to sequential requests
#58
dongmingli-Ben
closed
5 months ago
0
Feature/update with miss rate formula to global scheduler
#57
vikranth22446
closed
5 months ago
0
Update ref counter and better global scheduler handling of chains
#56
vikranth22446
closed
5 months ago
0
Feat/hc disagg
#55
jiange91
closed
5 months ago
0
Add virtualenv data loader
#54
dongmingli-Ben
closed
5 months ago
0
Add three data loader
#53
dongmingli-Ben
closed
5 months ago
0
Multi Experiment support
#52
vikranth22446
closed
5 months ago
0
Fix issues with lru cache
#51
vikranth22446
closed
5 months ago
0
Hot/cold integration and revert main back to before hot fix
#50
jiange91
closed
5 months ago
0
Fix lru cache evicted/non_evicted nodes
#49
vikranth22446
closed
5 months ago
0
update with fixed global lru
#48
vikranth22446
closed
6 months ago
0
Update the eviction policy + Initial work stealing policy
#47
vikranth22446
closed
6 months ago
1
Initial workload replication policy
#46
vikranth22446
closed
6 months ago
0
Add Multi domain tool loader
#45
vikranth22446
closed
6 months ago
0
Add Load based Memory Eviction
#44
vikranth22446
closed
6 months ago
0
Feat/hc disagg
#43
jiange91
closed
6 months ago
0
add support for vllm server via ssh
#42
dongmingli-Ben
closed
6 months ago
4
Feature/basic mem scheduler
#41
vikranth22446
closed
6 months ago
0
Feat/hc disagg
#40
jiange91
closed
6 months ago
1
Feat/add chunk prefill
#39
jiange91
closed
6 months ago
0
fix equation for LP
#38
jiange91
closed
6 months ago
0
Feature/add cache eviction
#37
vikranth22446
closed
6 months ago
1
Feat/hc disagg
#36
jiange91
closed
6 months ago
0
Update benchmarking utilities with more plotting
#35
vikranth22446
closed
6 months ago
0
cleanup workload handling
#34
vikranth22446
closed
6 months ago
0
Add support for greedy lp policy
#33
vikranth22446
closed
6 months ago
0
use flashinfer
#32
jiange91
closed
6 months ago
0
simple hot cold disagg
#31
jiange91
closed
6 months ago
0
Update global policy with better load management
#30
vikranth22446
closed
6 months ago
1
Feat/simulator
#29
jiange91
closed
6 months ago
0
Add basic load support to LP
#28
vikranth22446
closed
6 months ago
0
add missing profiling interface for model forwarding
#27
jiange91
closed
6 months ago
0
Remove old consistent hash router policy
#26
vikranth22446
closed
6 months ago
0
Next