streamText roundtrip result properties with past roundtrips

[woozar]

Description

I used the tool example with the calculator from the official website https://sdk.vercel.ai/docs/ai-sdk-core/agents and slightly adapted it to look like the provided code. Then I ran a prompt that used the tool to calculate some areas of some random parts.

After executing the code i get the following output on the console:

{ expression: '22 * 15' }
{ expression: '20.5 * 17' }
{ expression: '11 * 13' }
{ expression: '20 * 20' }
{ expression: '13 * 13' }
{ toolCalls: [], toolResults: [] }

Obviously the calculator was called for each part, but the toolCalls and toolResults in the onFinish are still empty.

Code example

  const result = await streamText({
    model: openai('gpt-40-2024-08-06'),
    temperature: 0,
    system: 'You are a helpful assistant and will support the user as good as you can.',
    messages: [{ role: 'user', content: 'Please calculate the area for the following parts. [{ "width": 22, "height": 15 }, { "width": 20.5, "height": 17 }, { "width": 11, "height": 13 }, { "width": 20, "height": 20 }, { "width": 13, "height": 13 }]' }] as CoreMessage[],
    tools: {
      calculator: tool({
        description:
          'A tool for evaluating mathematical expressions. ' +
          'Example expressions: ' +
          "'1.2 * (2 + 4.5)', '12.7 cm to inch', 'sin(45 deg) ^ 2'.",
        parameters: z.object({ expression: z.string() }),
        execute: async ({ expression }) => {
          console.log({ expression })
          return evaluate(expression)
        },
      }),
    },
    toolChoice: 'auto',
    maxToolRoundtrips: 10,
    onFinish({ toolCalls, toolResults }) {
      console.log({ toolCalls, toolResults })
    },
  })

I had to slightly adapt the code to put everything in one place and also replace the system prompt, which I am not allowed to publish (I am confident, that the system prompt is not the cause of this issue). Please forgive, if I did some syntax errors (or similar) during this process.

Additional context

Used packages:

• ai 3.3.26
• @ai-sdk/openai 0.0.55
• mathjs 13.1.1

[lgrammel]

onFinish only provides you with the tool calls & tool results from the last roundtrip. I would need to add roundtrips to onFinish / the result object similar to what generateText does.

Is using generateText an option?

[woozar]

We do need the stream, so it is not an option.

Also it is very irritating, that the toolCalls result which is described as "The tool calls that have been executed" does not contain all those calls. It took me quite a while to figure out "why the provided tool was never used", because it was used but not part of that list.

I am not sure if we need all the roundtrips (would not be the worst idea, probably) but for me personally, it would be sufficient to just get the toolCalls from all roundtrips and return the merged list

[leonho]

I'm experiencing a similar problem. I need a callback to save the tool results into client states using useChat and into the database using streamText, but both methods only provide the final assistant message.

[cmaycumber]

Same here.

Do we need to do something with onChunk and potentially parse each message out? Imagine this is a pretty common use case especially if you're saving the messages to a DB.

[lgrammel]

You can now access intermediate results using the steps property in onFinish