ray-project llmperf issues

ray-project / llmperf

LLMPerf is a library for validating and benchmarking LLMs

Apache License 2.0

467 stars 69 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

How to use llmperf to test ollama performance (TTFT, etc)

#58 alexhegit opened 1 week ago
0
fix: subsequent requests cannot be sent until 'num_concurrent_requests' requests have all finished in non-block mode

#57 llsj14 closed 1 week ago
1
Subsequent requests cannot be sent until 'num_concurrent_requests' requests have all finished

#56 llsj14 opened 1 week ago
2
Divide by zero: request_metrics[common_metrics.REQ_OUTPUT_THROUGHPUT] = num_output_tokens / request_metrics[common_metrics.E2E_LAT]

#55 yaronr opened 2 weeks ago
0
Improve benchmark tput by moving prompt preparation outside of loop

#54 gracehonv closed 1 week ago
1
Sagemaker client issue

#53 SuchethaChintha opened 3 weeks ago
0
Benchmark

#52 philschmid closed 3 weeks ago
0
Fix typo in README

#51 sparsh2 closed 3 weeks ago
1
DOC: add location to sonnet.txt and fix typo

#50 tuhinsharma121 opened 1 month ago
1
error in request_metrics dictionary implementation

#49 Durga2Dash opened 1 month ago
0
Vertex AI API needs to be updated.

#48 Durga2Dash opened 1 month ago
0
Since it does not have a 'setup.py' nor a 'setup.cfg', it cannot be installed in editable mode PEP 660

#47 biubiu3721 opened 1 month ago
0
fix: OpenAI Chat Completions Client Error with `delta` for specific providers and update Sonnet

#46 suptejas closed 1 month ago
1
Fix decoding throughput computation

#45 comaniac closed 2 months ago
0
unable to get the benchmark result

#44 dipshirajput opened 2 months ago
1
Blocking on pending requests despite block == false

#43 dacorvo opened 2 months ago
1
Add Hugging face client

#42 philschmid opened 3 months ago
1
Added Azure OpenAI endpoint support

#41 datlife opened 3 months ago
1
feat: Client Added for Predibase

#40 bhaba-ranjan opened 3 months ago
0
minor typo fix

#39 jimburtoft opened 3 months ago
0
HUGGINGFACE set

#38 capyun opened 3 months ago
0
Update README.md

#37 xieus opened 4 months ago
0
Added azureai client

#36 VindyaKonjarla opened 4 months ago
3
bug of counting output tokens

#35 irasin opened 4 months ago
0
llmperf not working for concurrent users

#34 nkanike07 opened 4 months ago
0
Bug: Hugging Face TGI not working

#33 ptrmayer opened 4 months ago
0
Concurrency level is not handled properly

#32 alexeykudinkin opened 5 months ago
0
Add memory bandwidth utilization metric

#31 mmcclean-aws opened 5 months ago
0
Handling custom codes arguments like trust_remote_code

#30 Akash08naik opened 5 months ago
0
usage for local models .

#29 Akash08naik opened 5 months ago
0
[Feature] Add Triton Inference Server support

#28 mo-hai closed 1 month ago
1
Fix typos and add caveats

#27 rickyyx closed 6 months ago
0
Fix Installation instructions in README.md

#26 mgoin closed 4 months ago
0
Fixed URL by adding version V1 prefix to OpenAI LLMPerf client

#25 drelu opened 6 months ago
0
Basic usage issue

#24 wangxingjun778 opened 6 months ago
0
Added json mode prob

#23 kouroshHakha closed 1 week ago
0
fix typo

#22 aniketmaurya closed 6 months ago
0
litellm serializable issue ?

#21 ishaan-jaff opened 6 months ago
0
Update llmperf.py

#20 kylehh closed 6 months ago
0
LLMPerfV2

#19 avnishn closed 6 months ago
2
MLC Integration and bug fixes

#18 MasterJH5574 closed 6 months ago
0
Crashing with large number of concurrent users

#17 francescov1 opened 7 months ago
0
Refactor endpoint config

#15 SumanthRH closed 6 months ago
3
Cleanup args handling

#14 SumanthRH closed 6 months ago
1
[Feature] Added TGI as supported framework

#13 jmcodero closed 7 months ago
1
Add support for 100+ LLMs - Anyscale, vertexai, ollama, perplexity, together ai, palm, openrouter

#12 ishaan-jaff closed 6 months ago
2
Does llmperf support measuring local disk models? What's the meaning of framework in llmperf.py line 355?

#11 zhangjiawei5911 closed 7 months ago
3
Update README.md

#10 eltociear closed 7 months ago
0
Are benchmark results released somewhere

#9 ogencoglu closed 6 months ago
5
Rename to llmperf

#8 waleedkadous closed 8 months ago
0