livekit / agents

Build real-time multimodal AI applications 🤖🎙️📹
https://docs.livekit.io/agents
Apache License 2.0
4.07k stars 427 forks source link

Cannot chain tool calls. Only 1 tool called per use query #1047

Open TheHassanShahzad opened 2 weeks ago

TheHassanShahzad commented 2 weeks ago

My application requires multiple tools to be used on conjunction with each other for a robotics application. I am testing the tool call ability of livekit before building my application but only 1 tool is called at a time. Even when explicitly modifying to system instructions to allow multiple tools to be called and modifying the tools to say this can be used in conjunction with other tools , nothing works

image

When i begin the agent and ask what is the current year it should first call the epoch tool and then call the get year tool to find the current year given the epoch

Unfortunately only the epoch tool gets called and the agent attempts to convert this to a year by itself without calling the get_year tool.

image

LangChain handles the agent tool calling perfectly and can chain tools exactly as intended. This however cannot.

image