issues
search
WukLab
/
preble
Stateful LLM Serving
Apache License 2.0
38
stars
6
forks
source link
Add support for greedy lp policy
#33
Closed
vikranth22446
closed
7 months ago
vikranth22446
commented
7 months ago
Implements a greedy lp policy for the scheduler.
Uses a single variable per iteration and iteratively decides which gpu to schedule by minimizing the max load to each gpu and the total recomputation cost
Implements a greedy lp policy for the scheduler.