ewantindale
commented
3 days ago Dealing with this issue as well when using Claude 3.5 Sonnet.
It starts off with a text response, then decides to call a function, and overwrites the text response with the function result. I would much rather have both the initial text response and the function result.
Description
I believe this could be expected behavior and simply a limitation? If so, please feel free to close this out (and please correct my understanding).
Reproduction is rather straightforward. Here's what I've been able to produce with consistently: 1) Model set to gpt-4-turbo or gpt-4o 2) Model provided a tool that produces nonsensical results (i.e. checkWeather, which accepts a location, but returns {prop1: 'Hello', prop2: 'World!'}) or something like that.
Now we need the model to call that nonsensical or broken function, and then prompt it to describe / explain the results of the function.
1) Prompt: "Please call checkWeather for Miami" 2) Model will call the tool, yielding the load state, and then returning the garbage data. 3) Prompt: "What was the response" | "How's it look" | etc. 4) Model will begin streaming a preface explaining that something seemed to have gone wrong, and that it will try to call the tool again. 5) Model will indeed call the tool again, in the same stream.
handleRender seems to be correctly detecting the new format, and runs the tool generator, however it does not seem to account for the possibility of having previous text-deltas, which (in my opinion / from my understanding) should use .append() to accomodate this transition by completing the text case and moving to the generator, rather than overwriting it.
Code example
No response
Additional context
Again there's a good chance I'm simply misunderstanding something about the core and this is currently a limitation or not how things are intended to be used.
I haven't spent a great deal of time looking into potential solutions for the reasons above (no point in spending the time if I'm misunderstanding or this is a known limitation with a planned fix), but I figured I would point out that this is certainly a possibility and one that users would probably find to feel like a bug.
Perhaps it's a straightforward as something like handling the transition case in the handleRender() conditional loop that is successfully handling the transition and overwrite in the first place to, as I said, .append() to the streamableUI rather than just having .update()'s => .done()? Perhaps it would be best to wait for the onFinish callback to facilitate sealing the preface before moving to the tool call deltas and generator?
Any thoughts or information are greatly appreciated! Apologies if I'm glossed over something and filed an uninformed report.