Error in program: list indices must be integers or slices, not str

[krrishdholakia]

The bug

Error in program: list indices must be integers or slices, not str

To Reproduce Give a full working code snippet that can be pasted into a notebook cell or python file. Make sure to include the LLM load step so we know which model you are using.

import guidance 
# connect to a chat model like GPT-4 or Vicuna
gpt4 = guidance.llms.OpenAI("gpt-3.5-turbo")
# vicuna = guidance.llms.transformers.Vicuna("your_path/vicuna_13B", device_map="auto")

experts = guidance('''
{{#system~}}
Imagine you're an AI fashion assistant. If a user is asking you for help, only suggest fashion items like clothing and accessories. Be concise and helpful. 
{{~/system}}

{{#user~}}
Imagine you're an AI fashion assistant. If a user is asking you for help, only suggest fashion items like clothing and accessories. Be concise and helpful. 
I want a response to the following question.
{{query}}
{{~/user}}

{{#assistant~}}
{{gen 'output' temperature=0.7 max_tokens=300}}
{{~/assistant}}
''', llm=gpt4)

experts(query='What should I wear to a wedding in the summer?')

System info (please complete the following information):

• running on google colab

-- I hit this error every 2nd run.

[krrishdholakia]

cc: @slundberg

[andysalerno]

Noticing the same. In _gen.py, seems like resp might sometimes be a list of responses, instead of a single response.

I have a hacky fix that works for me locally. in _gen.py I test if resp is a list.

        for resps in gen_obj:
            # <my change>
            if not isinstance(resps, list):
                resps = [resps]

            for resp in resps:
            # </my change>
                await asyncio.sleep(0) # allow other tasks to run
                #log("parser.should_stop = " + str(parser.should_stop))
                if parser.should_stop:
                    #log("Stopping generation")
                    break
                # log.debug("resp", resp)
                generated_value += resp["choices"][0]["text"]
                partial_output(resp["choices"][0]["text"])

edit: btw, I'm using a Transformer, locally hosted, not OpenAI. So seems like this impacts both scenarios.

[slundberg]

Thanks for report this! I just pushed a fix. It happened when trying to reuse the cache of a streamed result. I guess the unit tests didn't catch it because caching was turned off there...but that has been remedied now!