ray-project llmperf issues

ray-project / llmperf

LLMPerf is a library for validating and benchmarking LLMs

Apache License 2.0

471 stars 70 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

fix: subsequent requests cannot be sent until 'num_concurrent_requests' requests have all finished in non-block mode

#59 llsj14 opened 2 days ago
0
How to use llmperf to test ollama performance (TTFT, etc)

#58 alexhegit opened 1 week ago
0
fix: subsequent requests cannot be sent until 'num_concurrent_requests' requests have all finished in non-block mode

#57 llsj14 closed 1 week ago
1
Subsequent requests cannot be sent until 'num_concurrent_requests' requests have all finished

#56 llsj14 opened 2 weeks ago
4
Divide by zero: request_metrics[common_metrics.REQ_OUTPUT_THROUGHPUT] = num_output_tokens / request_metrics[common_metrics.E2E_LAT]

#55 yaronr opened 2 weeks ago
0
Improve benchmark tput by moving prompt preparation outside of loop

#54 gracehonv closed 2 weeks ago
1
Sagemaker client issue

#53 SuchethaChintha opened 3 weeks ago
0
Benchmark

#52 philschmid closed 3 weeks ago
0
Fix typo in README

#51 sparsh2 closed 3 weeks ago
1
DOC: add location to sonnet.txt and fix typo

#50 tuhinsharma121 opened 1 month ago
1
error in request_metrics dictionary implementation

#49 Durga2Dash opened 1 month ago
0
Vertex AI API needs to be updated.

#48 Durga2Dash opened 1 month ago
0
Since it does not have a 'setup.py' nor a 'setup.cfg', it cannot be installed in editable mode PEP 660

#47 biubiu3721 opened 1 month ago
0
fix: OpenAI Chat Completions Client Error with `delta` for specific providers and update Sonnet

#46 suptejas closed 2 months ago
1
Fix decoding throughput computation

#45 comaniac closed 2 months ago
0
unable to get the benchmark result

#44 dipshirajput opened 3 months ago
1
Blocking on pending requests despite block == false

#43 dacorvo opened 3 months ago
1
Add Hugging face client

#42 philschmid opened 3 months ago
1
Added Azure OpenAI endpoint support

#41 datlife opened 3 months ago
1
feat: Client Added for Predibase

#40 bhaba-ranjan opened 3 months ago
0
minor typo fix

#39 jimburtoft opened 3 months ago
0
HUGGINGFACE set

#38 capyun opened 3 months ago
0
Update README.md

#37 xieus opened 4 months ago
0
Added azureai client

#36 VindyaKonjarla opened 4 months ago
3
bug of counting output tokens

#35 irasin opened 4 months ago
0
llmperf not working for concurrent users

#34 nkanike07 opened 4 months ago
0
Bug: Hugging Face TGI not working

#33 ptrmayer opened 4 months ago
0
Concurrency level is not handled properly

#32 alexeykudinkin opened 5 months ago
0
Add memory bandwidth utilization metric

#31 mmcclean-aws opened 5 months ago
0
Handling custom codes arguments like trust_remote_code

#30 Akash08naik opened 5 months ago
0
usage for local models .

#29 Akash08naik opened 5 months ago
0
[Feature] Add Triton Inference Server support

#28 mo-hai closed 1 month ago
1
Fix typos and add caveats

#27 rickyyx closed 6 months ago
0
Fix Installation instructions in README.md

#26 mgoin closed 4 months ago
0
Fixed URL by adding version V1 prefix to OpenAI LLMPerf client

#25 drelu opened 6 months ago
0
Basic usage issue

#24 wangxingjun778 opened 6 months ago
0
Added json mode prob

#23 kouroshHakha closed 1 week ago
0
fix typo

#22 aniketmaurya closed 6 months ago
0
litellm serializable issue ?

#21 ishaan-jaff opened 7 months ago
0
Update llmperf.py

#20 kylehh closed 7 months ago
0
LLMPerfV2

#19 avnishn closed 7 months ago
2
MLC Integration and bug fixes

#18 MasterJH5574 closed 7 months ago
0
Crashing with large number of concurrent users

#17 francescov1 opened 7 months ago
0
Refactor endpoint config

#15 SumanthRH closed 7 months ago
3
Cleanup args handling

#14 SumanthRH closed 7 months ago
1
[Feature] Added TGI as supported framework

#13 jmcodero closed 7 months ago
1
Add support for 100+ LLMs - Anyscale, vertexai, ollama, perplexity, together ai, palm, openrouter

#12 ishaan-jaff closed 7 months ago
2
Does llmperf support measuring local disk models? What's the meaning of framework in llmperf.py line 355?

#11 zhangjiawei5911 closed 7 months ago
3
Update README.md

#10 eltociear closed 8 months ago
0
Are benchmark results released somewhere

#9 ogencoglu closed 6 months ago
5