search

ollama / ollama-js

Ollama JavaScript library
https://ollama.ai
MIT License
1.94k stars 136 forks source link

Mixing streaming and tool use #123

Open efritz opened 1 month ago

efritz commented 1 month ago

When I enable streaming it seems that the tool use JSON payload is sent as part of the content field of the ChatResponse value and tool_calls remains empty. Disabling streaming for the same model/prompt populates this array as I'd expect.

Is there a way to enable the same parsing with streaming enabled (or, alternatively a guide on how to consistently parse a tool use from the streaming model response)?

BruceMacD commented 1 month ago

Hey @efritz, this is a limitation on tool use with the latest version of Ollama. There is a fix open in the main repository now: https://github.com/ollama/ollama/pull/6066

I'll keep this issue open to track when this gets merged into the JavaScript SDK.