I have two other feature requests (#3451, #3208) that I've opened, both of which I will be closing in favor of this one as I believe this suggestion is the best way to handle this.
Currently when you use generateObject or generateText with a function call and the LLM responds with data that does not match your provided schema, the behavior is that the vercel AI SDK will throw a InvalidToolArgumentsError or AISDKError, which just returns the zod parsing error.
The issue here is that now the entire generateText or generateObject call is throwing a zod error, and the response from the provider is no longer accessible at all, even though that was technically a successful response. This is a problem for a few reasons
We lose access to what the LLM actually generated. Sometimes the response is recoverable / fixable, but we do not have an opportunity to handle those error cases.
Usage data is now lost (I know we could still save it with fetch intercepts, but this means we have to work with the data outside of the calling function scope)
I think of the vercel AI SDK as an extension of calling these providers, so it feels weird to me that a successful response from the LLM provider is completely inaccessible due to a zod parse error. I do agree that a generateText/generateObject should throw this error, but it should include the full LLM provider response.
In the case of #3208 when Groq throws an error directly from their API, the error that groq throws should be passed through the AISDKError.
If the error was thrown like this, that would be a huge, huge help.
try {
const {result} = await generateText({
...
});
} catch (e: InvalidToolArgumentsError) {
e.response
// Do something with the e.response
// Log usage data still
}
Feature Description
I have two other feature requests (#3451, #3208) that I've opened, both of which I will be closing in favor of this one as I believe this suggestion is the best way to handle this.
Currently when you use generateObject or generateText with a function call and the LLM responds with data that does not match your provided schema, the behavior is that the vercel AI SDK will throw a InvalidToolArgumentsError or AISDKError, which just returns the zod parsing error.
The issue here is that now the entire generateText or generateObject call is throwing a zod error, and the response from the provider is no longer accessible at all, even though that was technically a successful response. This is a problem for a few reasons
I think of the vercel AI SDK as an extension of calling these providers, so it feels weird to me that a successful response from the LLM provider is completely inaccessible due to a zod parse error. I do agree that a generateText/generateObject should throw this error, but it should include the full LLM provider response.
In the case of #3208 when Groq throws an error directly from their API, the error that groq throws should be passed through the AISDKError.
If the error was thrown like this, that would be a huge, huge help.
Use Cases
No response
Additional context
No response