Error in generating tokens with example_vllm.py

fmperf-project / fmperf

Cloud Native Benchmarking of Foundation Models

Apache License 2.0

21 stars 10 forks source link

Closed GhaziSyed closed 2 weeks ago

GhaziSyed commented 4 months ago

looks like there is 'NoneType' object is not subscriptable error that needs to be fixed.

how to reproduce: run the example_vllm.py file, and check the ./examples/pod_log_response.txt to check for this error
the environment on which this is tested: head points to the commit f22b2d1 note that we are hitting the server with requests in the expected manner

GhaziSyed commented 2 weeks ago

This was resolved.