search

ray-project / llmperf

LLMPerf is a library for validating and benchmarking LLMs
Apache License 2.0
470 stars 69 forks source link

Sagemaker client issue #53

Open SuchethaChintha opened 3 weeks ago

SuchethaChintha commented 3 weeks ago

when i am executing token_benchmark_ray.py we are getting below error File "token_benchmark_ray.py", line 456, in run_token_benchmark( File "token_benchmark_ray.py", line 297, in run_token_benchmark summary, individual_responses = get_token_throughput_latencies( File "token_benchmark_ray.py", line 111, in get_token_throughput_latencies request_metrics[common_metrics.INTER_TOKEN_LAT] /= num_output_tokens TypeError: unsupported operand type(s) for /=: 'list' and 'int' (SageMakerClient pid=15473) Warning Or Error: 'SageMakerRuntime' object has no attribute 'invoke_endpoint_with_response_stream' (SageMakerClient pid=15473) None