OpenAI Model tries to call unavailable tool `multi_tool_use.parallel`

aryanbhasin commented 2 weeks ago

Description

When calling generateText with a newer OpenAI model like gpt-4o and one or more tools, the model - somewhat sporadically - throws the following error:

NoSuchToolError [AI_NoSuchToolError]: Model tried to call unavailable tool 'multi_tool_use.parallel'. Available tools: {my_tool_name}.

Longer stack --


[Symbol(vercel.ai.error.AI_NoSuchToolError)]: true |  
[Symbol(vercel.ai.error)]: true, |  
availableTools: [ 'escalate_customer_request' ], |  
toolName: 'multi_tool_use', |  
cause: undefined, |  
at async EventEmitter.<anonymous> (/app/dist/services/wppConnect.js:186:26) { |  
at async processMessage (/app/dist/src/whatsapp.js:103:32) |  
at async handleOpenAIResponse (/app/dist/src/ai.js:83:24) |  
at async /app/node_modules/.pnpm/ai@3.3.7_openai@4.47.1_react@18.3.1_svelte@4.2.18_vue@3.4.37_zod@3.23.8/node_modules/ai/dist/index.js:338:22 |  
at process.processTicksAndRejections (node:internal/process/task_queues:95:5) |  
at fn (/app/node_modules/.pnpm/ai@3.3.7_openai@4.47.1_react@18.3.1_svelte@4.2.18_vue@3.4.37_zod@3.23.8/node_modules/ai/dist/index.js:2395:90) |  
at Array.map (<anonymous>) |  
at /app/node_modules/.pnpm/ai@3.3.7_openai@4.47.1_react@18.3.1_svelte@4.2.18_vue@3.4.37_zod@3.23.8/node_modules/ai/dist/index.js:2396:30 |  
at parseToolCall (/app/node_modules/.pnpm/ai@3.3.7_openai@4.47.1_react@18.3.1_svelte@4.2.18_vue@3.4.37_zod@3.23.8/node_modules/ai/dist/index.js:2242:11)

This is possibly an issue with the underling model itself, see the entire discussion here on the OpenAI forum. Some solutions explored on the forum were -

Prompting to ask the LLM to not call multiple tools at once (unreliable)
Checking if tool result has multi_tool_use.parallel, and either abandoning the request or trying it again
Prefix-ing tool names with functions.

On the AI SDK's behalf, a possible remedy would be supporting the parallel_tool_calls: false flag which is currently not an option under generateText. Read more on that flag here

Code example

import { generateText } from 'ai';
import { openai } from '@ai-sdk/openai';

const result = await generateText({
      model: openai('gpt-4o-2024-08-06'),
      system: 'You are a ...',
      messages: [...chatHistory, { role: 'user', content: 'How do I ...' }],
      tools: [tool({
          description: '...',
          parameters: ...
      })],
});

Additional context

This error happens unpredictably - sometimes the tool gets called successfully and sometimes the model throws this error. The error gets thrown even if we pass one tool, which suggests there are probably other tools (like web search) that OpenAI puts under the multi_tool_use wrapper

lgrammel commented 2 weeks ago

You can disable parallel tool calls, see https://sdk.vercel.ai/providers/ai-sdk-providers/openai#chat-models

aryanbhasin commented 2 weeks ago

You can disable parallel tool calls, see https://sdk.vercel.ai/providers/ai-sdk-providers/openai#chat-models

Ah amazing, thank you. I looked for that flag in generateText but makes sense to configure it in the model provider itself.

vercel / ai