[Bug]: Mising tool outputss Error when using Azure OpenAI Assistants, when the Assistant planner calls the same tool multiple times in a run

[NikhilB95]

Language

Python

Version

latest

Description

When using Azure OpenAI Assistants, with custom tools, the teams bot raises a error (shown below) in specific situations when the same tool is called multiple times.

Error: Traceback (most recent call last): File "/tmp/8dc83d67fdc82b2/antenv/lib/python3.11/site-packages/botbuilder/core/bot_adapter.py", line 174, in run_pipeline return await self._middleware.receive_activity_with_status( File "/tmp/8dc83d67fdc82b2/antenv/lib/python3.11/site-packages/botbuilder/core/middleware_set.py", line 69, in receive_activity_with_status return await self.receive_activity_internal(context, callback) File "/tmp/8dc83d67fdc82b2/antenv/lib/python3.11/site-packages/botbuilder/core/middleware_set.py", line 79, in receive_activity_internal return await callback(context) File "/tmp/8dc83d67fdc82b2/antenv/lib/python3.11/site-packages/teams/app.py", line 663, in on_turn await self._start_long_running_call(context, self._on_turn) File "/tmp/8dc83d67fdc82b2/antenv/lib/python3.11/site-packages/teams/app.py", line 813, in _start_long_running_call return await func(context) File "/tmp/8dc83d67fdc82b2/antenv/lib/python3.11/site-packages/teams/app.py", line 756, in _on_turn is_ok = await self._ai.run(context, state) File "/tmp/8dc83d67fdc82b2/antenv/lib/python3.11/site-packages/teams/ai/ai.py", line 187, in run return await self.run(context, state, started_at, step) File "/tmp/8dc83d67fdc82b2/antenv/lib/python3.11/site-packages/teams/ai/ai.py", line 187, in run return await self.run(context, state, started_at, step) File "/tmp/8dc83d67fdc82b2/antenv/lib/python3.11/site-packages/teams/ai/ai.py", line 143, in run plan = await self.planner.continue_task(context, state) File "/tmp/8dc83d67fdc82b2/antenv/lib/python3.11/site-packages/teams/ai/planners/assistants_planner.py", line 187, in continue_task return await self._submit_action_results(state) File "/tmp/8dc83d67fdc82b2/antenv/lib/python3.11/site-packages/teams/ai/planners/assistants_planner.py", line 279, in _submit_action_results run = await self._client.beta.threads.runs.submit_tool_outputs( File "/tmp/8dc83d67fdc82b2/antenv/lib/python3.11/site-packages/openai/resources/beta/threads/runs/runs.py", line 2979, in submit_tool_outputs return await self._post( File "/tmp/8dc83d67fdc82b2/antenv/lib/python3.11/site-packages/openai/_base_client.py", line 1790, in post return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls) File "/tmp/8dc83d67fdc82b2/antenv/lib/python3.11/site-packages/openai/_base_client.py", line 1493, in request return await self._request( File "/tmp/8dc83d67fdc82b2/antenv/lib/python3.11/site-packages/openai/_base_client.py", line 1584, in _request raise self._make_status_error_from_response(err.response) from None openai.BadRequestError: Error code: 400 - {'error': {'message': "Expected tool outputs for call_ids ['call_dSFAdfcF9CLsB6LutGJleiFJ', 'call_kxqGFm4LegeYA80wdG4nX0q4', 'call_RzLxCptvGVXCD298Klh170xX'], got ['call_RzLxCptvGVXCD298Klh170xX']", 'type': 'invalid_request_error', 'param': None, 'code': None}}

Reproduction Steps

1. Setup custom tools via function calling
2. Prompt the AssistantPlanner to call the same tool multiple times in a single prompt
3. Assistant Raises a error that tool outputs are missing

I believe the error stems from the _generate_plan_from_tools function in assistant_planner.py
Here, the tool_map Dict is initiated with the dictionary keys as the function name. This raises a issue where the same function output is overwritten in tools_map in case of multiple runs of the same function.
This further effects _submit_action_results, where the error stems from.

[lilyydu]

Hi @NikhilB95, do you mind indenting the stack trace line-by-line so that it is easier to see?

Do you know which call_id is associated to the function you are trying to call multiple times?

I see this error at the bottom- it looks like the outputs of the first two calls were not returned? Error code: 400 - {'error': {'message': "Expected tool outputs for call_ids ['call_dSFAdfcF9CLsB6LutGJleiFJ', 'call_kxqGFm4LegeYA80wdG4nX0q4', 'call_RzLxCptvGVXCD298Klh170xX'], got ['call_RzLxCptvGVXCD298Klh170xX']", 'type': 'invalid_request_error', 'param': None, 'code': None}}

[lilyydu]

Closing due to inactivity, please re-open if support is required.