issues
search
CentML
/
flexible-inference-bench
A modular, extensible LLM inference benchmarking framework that supports multiple benchmarking frameworks and paradigms.
Apache License 2.0
5
stars
0
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
Unclear how to use datasets other than random
#73
vvagias
opened
1 week ago
1
[DOCS] update docs
#72
johncalesp
closed
3 weeks ago
0
`performance.py` does not produce results
#71
vadiklyutiy
closed
3 weeks ago
1
`inference-benchmark` not found
#70
vadiklyutiy
closed
3 weeks ago
1
Fix README.md misprint
#69
vadiklyutiy
closed
3 weeks ago
1
[FIX] Solve TPOT error
#68
johncalesp
closed
4 weeks ago
0
small patch
#67
johncalesp
closed
1 month ago
0
Bugfix for `performance.py` reporting incorrect TPOT
#66
benchislett
opened
1 month ago
5
use Union instead of | for 3.9 compatibility
#65
xinli-centml
closed
1 month ago
0
Data Generation of random negative number throws error occasionally
#64
vvagias
opened
1 month ago
3
[ARGS] add subparsers
#63
johncalesp
closed
1 month ago
2
[Postprocessors] Update itl.py to savefig before showing
#62
andoorve
closed
1 month ago
0
Create LICENSE
#61
xinli-centml
closed
2 months ago
0
[PLOT] Changes to ttft plot
#60
johncalesp
closed
3 months ago
3
[BACKEND] Tested agains TRT
#59
johncalesp
closed
3 months ago
5
[DEPENDENCY] add pydantic
#58
johncalesp
closed
3 months ago
0
Connect flexible inference bench with CServe deloyment config
#57
xinli-centml
opened
3 months ago
0
[DOCS] add example for vllm
#56
johncalesp
closed
3 months ago
1
[Bugfix] Rename main.py to __main__.py
#55
andoorve
closed
3 months ago
0
TensorRT-LLM Support
#54
andoorve
opened
3 months ago
1
[CHECKLIST] Checklist before release to public
#53
johncalesp
closed
1 month ago
4
[UNIT TESTS] Change github action and test definition
#52
johncalesp
closed
4 months ago
0
Get 400 error from vLLM
#51
andoorve
closed
4 months ago
2
[UNIT TEST] unit test for backend enpoints
#50
johncalesp
closed
4 months ago
4
Fix parsing benchmark seed when seed is 0
#49
wzhao18
closed
4 months ago
1
[GRAPHS] Plot for ttft
#48
johncalesp
closed
4 months ago
1
[IMPROVEMENTS] Early stopping and verbose output
#47
johncalesp
closed
4 months ago
4
[DOCS] updated README
#46
johncalesp
closed
5 months ago
0
Change name to flexible-inference-benchmark
#45
andoorve
closed
5 months ago
2
[ADDED ARGS] added new args: stream, cookies
#44
johncalesp
closed
5 months ago
2
[API] Add ignore eos and some cleanup
#43
andoorve
closed
5 months ago
0
Latency variable is not initialized in async_request_openai_completions
#42
atokayev
opened
5 months ago
2
[WORKLOAD TYPES]
#41
johncalesp
closed
3 months ago
7
Validate the way performance.py calculates benchmarks
#40
andoorve
closed
4 months ago
0
Random input sometimes generates no tokens
#39
andoorve
closed
4 months ago
2
Add ignore_eos argument
#38
andoorve
opened
5 months ago
1
[OUTPUT FILENAME AND VALIDATOR]
#37
johncalesp
closed
5 months ago
1
[Client] minor enhacements to the script and allow benchmarker to bypass ssl verification for CentmL platform
#36
xinli-centml
closed
5 months ago
0
[Usage] make output file required?
#35
xinli-centml
closed
4 months ago
1
[Usage] Why do we need both host_port and base_url?
#34
xinli-centml
closed
4 months ago
1
Update README.md
#33
atokayev
closed
5 months ago
0
Make seed different each time by default
#32
andoorve
closed
5 months ago
0
Cookie Argument
#31
andoorve
closed
4 months ago
1
Database Input
#30
andoorve
closed
4 months ago
1
Random Seed
#29
andoorve
closed
5 months ago
1
Added arg examples and reintroduced cserve backend after bug fix
#28
andoorve
closed
5 months ago
0
Early Fail
#27
andoorve
closed
4 months ago
1
[Post-process] Add vllm performance postprocessor
#26
andoorve
closed
5 months ago
0
[Post-Processing] Add example itl postprocessor
#25
andoorve
closed
5 months ago
0
[Postprocessors] Add example ITL postprocessor
#24
andoorve
closed
5 months ago
0
Next