ray-project / llmperf

LLMPerf is a library for validating and benchmarking LLMs
Apache License 2.0
470 stars 69 forks source link

fix: subsequent requests cannot be sent until 'num_concurrent_requests' requests have all finished in non-block mode #57

Closed llsj14 closed 1 week ago

llsj14 commented 1 week ago

issues

https://github.com/ray-project/llmperf/issues/43 https://github.com/ray-project/llmperf/issues/56

Summary

llsj14 commented 1 week ago

This commit is not enough to fix the problem, so close this PR.