search

vercel / ai

Build AI-powered applications with React, Svelte, Vue, and Solid
https://sdk.vercel.ai/docs
Other
10.31k stars 1.55k forks source link

Return the execution result for the provider execution tools #3439

Open minpeter opened 1 month ago

minpeter commented 1 month ago

Feature Description

Search Grounding in Google Vertex AI, codeExecution in Google Generative AI, and built-in tools in Friendli Provider provide tools like search, code execution.

These tools reduce the need to define and worry about repeated tool definitions and behaviors, and reduce the latency of tool execution.

If we call these kinds of tools provider-executed tools, the current AI sdk does not allow us to know the execution status and results of the tools executed in the provider.

I think it would be a great improvement to be able to know whether these provider-executed tools are executed and to be able to check the results.

As a developer of the Friendli community provider, I tried to implement this, but ai/core does not provide an appropriate way to do so.

There is an issue related to this.

What makes it hard to leave provider execution tool results...?

LanguageModelV1StreamPart structure does not have tool-result. At least I expect that tool-result will be generated here and if it is properly parsed, it will be possible to implement it.

Use Case

Even if you use the above function, you can show the tool call status to the user's AI.

Additional context

I'm not sure if this is a legitimate restriction. I just hope it's reviewed positively. In the case of claude's compute vision, which is a provider-defined tool, it is to define the schema of the tool, but the above Search Grounding It will be a cool feature.. :)

kjy3309 commented 1 month ago

I strongly support this feature request. I'm currently facing the same issue while building a backend with LangGraph and using ai-sdk. The ability to access provider-executed tool results would be extremely valuable for our development workflow.

lgrammel commented 1 month ago

This is a important request and something I've been thinking about. It'll have to come post 4.0, i.e. 2nd half of November or early December, since we have started a feature freeze.

minpeter commented 1 month ago

I'm really looking forward to the release of AI SDK version 4.0. 🤩

Linnore commented 1 week ago

Same request. This feature will be extremely useful for applications that backended by LLM orchestrations liken LangGraph and CrewAI!