ray-project llmperf issues

ray-project / llmperf

LLMPerf is a library for validating and benchmarking LLMs

Apache License 2.0

658 stars 113 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

error

#79 TomerG711 closed 4 weeks ago
0
Support other scenarios

#78 yw-furiosa opened 1 month ago
0
how to test vllm interface

#77 ZTurboX opened 1 month ago
0
Using `max_concurrency` or AsyncActor?

#76 chiragjn opened 1 month ago
0
Wrong generated prompt length

#75 PietroFerr opened 1 month ago
0
Refactor to use `yarl` package for safe URL manipulation

#74 rapsealk opened 1 month ago
1
Feat/trust remote code

#73 jouDance opened 1 month ago
0
Adjust chat completion request message order

#72 ohm314 closed 1 month ago
1
Raise exception when errors

#71 markVaykhansky closed 2 months ago
0
Error while using pip

#70 avi7611 opened 2 months ago
1
Contributions guidelines

#69 miloszwatroba opened 2 months ago
0
Use appropriate tokenizer

#68 markVaykhansky closed 2 months ago
0
Azure OpenAi Endpoint Support

#67 gujju62 opened 3 months ago
0
Add local triton client

#66 protagohhz closed 3 months ago
0
Update openai_chat_completions_client.py

#65 jmhessel closed 3 months ago
0
fix issue #63

#64 huangdi614 opened 3 months ago
0
When max-num-completed-requests is not divisible by num-concurrent-requests, an error will occur.

#63 huangdi614 opened 3 months ago
0
fix typo

#62 zhimin-z opened 3 months ago
0
Fixes `Divide by zero` error

#61 e3oroush opened 3 months ago
0
Update README from LLMPerf to BeFOri

#60 ori-cfowler closed 4 months ago
0
fix: subsequent requests cannot be sent until 'num_concurrent_requests' requests have all finished in non-block mode

#59 llsj14 opened 5 months ago
5
How to use llmperf to test ollama performance (TTFT, etc)

#58 alexhegit opened 5 months ago
1
fix: subsequent requests cannot be sent until 'num_concurrent_requests' requests have all finished in non-block mode

#57 llsj14 closed 5 months ago
1
Subsequent requests cannot be sent until 'num_concurrent_requests' requests have all finished

#56 llsj14 opened 5 months ago
5
Divide by zero: request_metrics[common_metrics.REQ_OUTPUT_THROUGHPUT] = num_output_tokens / request_metrics[common_metrics.E2E_LAT]

#55 yaronr opened 5 months ago
7
Improve benchmark tput by moving prompt preparation outside of loop

#54 gracehonv closed 5 months ago
1
Sagemaker client issue

#53 SuchethaChintha opened 5 months ago
1
Benchmark

#52 philschmid closed 6 months ago
0
Fix typo in README

#51 sparsh2 closed 5 months ago
1
DOC: add location to sonnet.txt and fix typo

#50 tuhinsharma121 opened 6 months ago
1
error in request_metrics dictionary implementation

#49 Durga2Dash opened 6 months ago
0
Vertex AI API needs to be updated.

#48 Durga2Dash opened 6 months ago
0
Since it does not have a 'setup.py' nor a 'setup.cfg', it cannot be installed in editable mode PEP 660

#47 focusunsink opened 6 months ago
1
fix: OpenAI Chat Completions Client Error with `delta` for specific providers and update Sonnet

#46 suptejas closed 7 months ago
1
Fix decoding throughput computation

#45 comaniac closed 7 months ago
0
unable to get the benchmark result

#44 dipshirajput opened 8 months ago
1
Blocking on pending requests despite block == false

#43 dacorvo opened 8 months ago
1
Add Hugging face client

#42 philschmid opened 8 months ago
2
Added Azure OpenAI endpoint support

#41 datlife opened 8 months ago
1
feat: Client Added for Predibase

#40 bhaba-ranjan opened 8 months ago
0
minor typo fix

#39 jimburtoft opened 8 months ago
0
HUGGINGFACE set

#38 capyun opened 8 months ago
0
Update README.md

#37 xieus opened 9 months ago
0
Added azureai client

#36 VindyaKonjarla opened 9 months ago
3
bug of counting output tokens

#35 irasin opened 9 months ago
0
llmperf not working for concurrent users

#34 nkanike07 opened 9 months ago
0
Bug: Hugging Face TGI not working

#33 ptrmayer opened 9 months ago
0
Concurrency level is not handled properly

#32 alexeykudinkin opened 10 months ago
0
Add memory bandwidth utilization metric

#31 mmcclean-aws opened 10 months ago
0
Handling custom codes arguments like trust_remote_code

#30 Akash08naik opened 10 months ago
0