-
Tools/functions should be supported in streaming mode. Currently, they work only in sync mode.
-
This seems like a current limitation in Ollama itself -- see https://github.com/ollama/ollama/issues/5796 and https://github.com/ollama/ollama/pull/6452
```python
from chatlas import ChatOllama
chat…
-
Tool support has been implemented for `OllamaChatModel` in #1558.
We need to implement the same for `OllamaStreamingChatModel`.
Implementation in `OpenAiStreamingChatModel` and `OpenAiStreamingR…
-
### Question Validation
- [X] I have searched both the documentation and discord for an answer.
### Question
I have an LLM model (specifically from Anthropic) and I want to use the native tools fro…
-
### What happened?
It's hard to reproduce, but if the tool call have lot of data (Like generating a website) everything it's streamed normally until a point where no more chunk are received until the…
-
(Sorry if this isn't the right place to report this, I wasn't sure).
I'm trying to switch from gpt-4o to claude-3.5-sonnet in an app I'm building, but high streaming tool latency is preventing me f…
-
When I enable streaming it seems that the tool use JSON payload is sent as part of the `content` field of the `ChatResponse` value and `tool_calls` remains empty. Disabling streaming for the same mode…
-
When using the Anthropic Agent with `streaming: true`, you get the error: `"Anthropic does not support streaming` . If you remove the code block https://github.com/run-llama/LlamaIndexTS/blob/45bcf512…
-
I used the [hermes_tool_parser.py](https://github.com/vllm-project/vllm/blob/main/vllm/entrypoints/openai/tool_parsers/hermes_tool_parser.py) as `tool-parser-plugin` and registered the parser as `herm…
-
### What happened?
I'm using the docker version, with the version `v1.48.19-stable`. After a tool call I got the following error
### Relevant log output
```shell
litellm-1 | 12:51:23 - LiteLLM:DEB…