search

guidance-ai / guidance

A guidance language for controlling large language models.
MIT License
19.05k stars 1.04k forks source link

gpt-3.5-turbo-16k azure call is failing #321

Open sidhartha-sankar opened 1 year ago

sidhartha-sankar commented 1 year ago

The bug The call with azure key for gpt-3.5-turbo-16k fails for list index out of range error. ''''''''''''''''''''''''''''''''''''' 3-07-18 04:22:00,419 INFO [qna_logic.py:51] [ds_models.all_guidance.qna_logic] [1dd58ad8-d1c2-4367-bd4e-8e99681d2a82][SESSION: abcd-session][QUESTIONID: question_1] Time taken for embed_query_timing :0.3142378330230713 2023-07-18 04:22:00,419 INFO [qna_logic.py:51] [ds_models.all_guidance.qna_logic] [1dd58ad8-d1c2-4367-bd4e-8e99681d2a82][SESSION: abcd-session][QUESTIONID: question_1] Completed vector search, switching to Azure 16k call 2023-07-18 04:22:00,480 INFO [qna_logic.py:51] [ds_models.all_guidance.qna_logic] [1dd58ad8-d1c2-4367-bd4e-8e99681d2a82][SESSION: abcd-session][QUESTIONID: question_1] Time taken for answer_stream_start :0.06079912185668945 Traceback (most recent call last): File "/home/raamb/workspace/docker-privateblok-api/env/lib/python3.10/site-packages/guidance/_program_executor.py", line 109, in run await self.visit(self.parse_tree, VariableStack([self.program._variables], self)) File "/home/raamb/workspace/docker-privateblok-api/env/lib/python3.10/site-packages/guidance/_program_executor.py", line 559, in visit visited_children.append(await self.visit(child, variable_stack, inner_next_node, inner_next_next_node, inner_prev_node, node, parent_node)) File "/home/raamb/workspace/docker-privateblok-api/env/lib/python3.10/site-packages/guidance/_program_executor.py", line 524, in visit command_output = await command_function(positional_args, named_args) File "/home/raamb/workspace/docker-privateblok-api/env/lib/python3.10/site-packages/guidance/library/_assistant.py", line 13, in assistant return await role(role_name="assistant", hidden=hidden, _parser_context=_parser_context, kwargs) File "/home/raamb/workspace/docker-privateblok-api/env/lib/python3.10/site-packages/guidance/library/_role.py", line 17, in role new_content += await parser.visit( File "/home/raamb/workspace/docker-privateblok-api/env/lib/python3.10/site-packages/guidance/_program_executor.py", line 559, in visit visited_children.append(await self.visit(child, variable_stack, inner_next_node, inner_next_next_node, inner_prev_node, node, parent_node)) File "/home/raamb/workspace/docker-privateblok-api/env/lib/python3.10/site-packages/guidance/_program_executor.py", line 559, in visit visited_children.append(await self.visit(child, variable_stack, inner_next_node, inner_next_next_node, inner_prev_node, node, parent_node)) File "/home/raamb/workspace/docker-privateblok-api/env/lib/python3.10/site-packages/guidance/_program_executor.py", line 266, in visit visited_children = [await self.visit(child, variable_stack, next_node, next_next_node, prev_node, node, parent_node) for child in node] File "/home/raamb/workspace/docker-privateblok-api/env/lib/python3.10/site-packages/guidance/_program_executor.py", line 266, in visited_children = [await self.visit(child, variable_stack, next_node, next_next_node, prev_node, node, parent_node) for child in node] File "/home/raamb/workspace/docker-privateblok-api/env/lib/python3.10/site-packages/guidance/_program_executor.py", line 379, in visit command_output = await command_function(positional_args, **named_args) File "/home/raamb/workspace/docker-privateblok-api/env/lib/python3.10/site-packages/guidance/library/_gen.py", line 172, in gen new_text = resp["choices"][0].get("text", "") IndexError: list index out of range

Error in program: list index out of range ''''''''''''''''''''''''''''''''''''''''''''''''''

To Reproduce Any guidance call with azure 16k System info (please complete the following information):

A possible fix has been provided as part of : https://github.com/microsoft/guidance/pull/319

aslisabanci commented 1 year ago

Possibly related? https://github.com/guidance-ai/guidance/issues/340