Reported on slack. Regression from function calling.
FinishAction with source=agent is not the last thing that happens when running with UI or restoring session in CLI, but our backend doesn't give it a tool response. CodeAct "forgets" it, which makes other things not play nice.
Note: it happens with native function calling, but it probably happens with non-native too after we started converting them back and forth.
IMPORTANT:
find in the codebase and read codeact_agent.py, function_calling.py, fn_call_converter.py, llm.py.
the problem is in codeact agent, the other files are only for context on the processing that happens in the agent
the FinishAction with source=user doesn't have a tool response, because it just returns (which will end execution). Find and read the relevant code in codeact that checks for this action. Hint: at the start of step().
the FinishAction with source=agent is a tool to execute, but its execution is the same: it returns, and it doesn't get a tool response.
With this context, fix this:
either add a tool response, correctly linked to the tool call by id
or make sure its thought is sent to the llm as if it was a message action, and its tool id is never sent
note that in get_action_message() there are other odd cases; we can add these two.
IMPORTANT INFO:
We are fixing an AI agent. The agent sends multiple messages to an LLM API. That API and how it works are impossible for us to change! It is important that you do NOT hallucinate their behavior.
One requirement for tools is, for example, that a message that is a tool call, has a tool id, and is immediately followed by a tool response, which is linked to the previous message by tool id.
Is there an existing issue for the same bug?
Describe the bug and reproduction steps
Reported on slack. Regression from function calling.
FinishAction with
source=agent
is not the last thing that happens when running with UI or restoring session in CLI, but our backend doesn't give it a tool response. CodeAct "forgets" it, which makes other things not play nice.Note: it happens with native function calling, but it probably happens with non-native too after we started converting them back and forth.
IMPORTANT:
With this context, fix this:
IMPORTANT INFO: We are fixing an AI agent. The agent sends multiple messages to an LLM API. That API and how it works are impossible for us to change! It is important that you do NOT hallucinate their behavior.
One requirement for tools is, for example, that a message that is a tool call, has a tool id, and is immediately followed by a tool response, which is linked to the previous message by tool id.
Additional Context
It will be addressed by this refactoring or before.