Closed krrishdholakia closed 1 year ago
cc: @slundberg
Noticing the same. In _gen.py, seems like resp
might sometimes be a list of responses, instead of a single response.
I have a hacky fix that works for me locally. in _gen.py I test if resp
is a list.
for resps in gen_obj:
# <my change>
if not isinstance(resps, list):
resps = [resps]
for resp in resps:
# </my change>
await asyncio.sleep(0) # allow other tasks to run
#log("parser.should_stop = " + str(parser.should_stop))
if parser.should_stop:
#log("Stopping generation")
break
# log.debug("resp", resp)
generated_value += resp["choices"][0]["text"]
partial_output(resp["choices"][0]["text"])
edit: btw, I'm using a Transformer, locally hosted, not OpenAI. So seems like this impacts both scenarios.
Thanks for report this! I just pushed a fix. It happened when trying to reuse the cache of a streamed result. I guess the unit tests didn't catch it because caching was turned off there...but that has been remedied now!
The bug
Error in program: list indices must be integers or slices, not str
To Reproduce Give a full working code snippet that can be pasted into a notebook cell or python file. Make sure to include the LLM load step so we know which model you are using.
System info (please complete the following information):
-- I hit this error every 2nd run.