Let AI use multiple times tools before answering

[louis030195]

Feature Description

"Can you tell me the weather tomorrow and on thursday" -> Now AI would make a query and then answer the question

Would be cool if the AI could make multiple requests like one for tomorrow and one for thursday. I very often implement this use case with LLM (been using LLM since gpt-1)

example use case: https://github.com/louis030195/screen-pipe/blob/57dc0fce2f10cbd1053f2f3fac3cd69d4ae194ba/examples/ts/rag-over-your-life-in-obsidian/screenpipe.js#L118

Use Case

see above

Additional context

I've seen the "agent" parts in your doc. didn't get it

I understand that sometimes you want a loop with LLM to let them iterate, like here: https://github.com/llm-edge/hal-9100/blob/5acaf971309f344b6cb25572ae0f71424872ffdc/hal-9100-core/src/code_interpreter.rs#L70

but didnt understand the UX at all, think it's different than what i ask here though

[lgrammel]

There are different cases:

a) you want the llm to use tools, the tools should be executed automatically, the llm should use the tool result, and it all should be flexible. you don't need to stream to the user. then you can use maxToolRoundtrips: https://sdk.vercel.ai/docs/ai-sdk-core/agents

b) you have a fixed script, e.g. force the llm to generate a tool calls, then do something with it, then invoke the llm with a different structure: use generateText / generateObject and Javascript code to write out that fixed setup

c) you need to have an interactive chatbot use tools: use streamText + useChat in combination: https://sdk.vercel.ai/docs/ai-sdk-ui/chatbot-with-tool-calling

I'm not sure this covers what you are looking for. I looked at your code but didn't fully understand what you want to achieve. Would you mind writing down an example use case step by step?

[kdawgwilk]

@lgrammel unfortunately a is what we are going for but we are also using streamText which does not support maxToolRoundtrips 😞 Looking forward to #1943 being implemented so we don't have to hack together workarounds to support tool calling with streaming

[louis030195]

@lgrammel thanks!

actually i found a solution which is perfect for my use case:

https://github.com/louis030195/screen-pipe/blob/dcd5773eb74c32f4a7797e3f7a22a01376acd0a0/examples/ts/vercel-ai-chatbot/lib/chat/actions.tsx#L114

basically i use z.array for parameters so the LLM generate multiple queries I execute in parallel then pass to the LLM (kinda map reduce style)

it works MUCH better than a single query

(the LLM generate multiple search queries in parallel in my use case)

problem solved for me

also i tend to dislike hacks like what langchain does (or other popular repo that don't pass the test of time), LLMs will converge towards doing the whole reasoning as a single API call

[cosbgn]

@lgrammel this is not completed yet right? can we re-open this?