Open Clad3815 opened 2 months ago
can you share a sample curl for repro? @Clad3815
I'm not using curl, just OpenAI sdk with litellm setup as proxy (On docker).
This config work:
- model_name: 'llava:7b'
litellm_params:
model: 'ollama/llava:7b'
api_base: 'http://host.docker.internal:11434'
But not this one:
- model_name: 'llava:7b'
litellm_params:
model: 'ollama_chat/llava:7b'
api_base: 'http://host.docker.internal:11434'
I don't know what I can provide more, any request will result in that
facing the same issue.
My fix was to instead regular ollama not ollama_chat:
litellm --model ollama/llama3.1:latest --drop_params
I am facing the same issue with the Ollama function call using the Llava model. When I use ollama/llava, I get the error: KeyError: 'name': function_call["name"]. However, when I switch to ollama_chat/llava, I encounter a different error: json: cannot unmarshal array into Go struct field ChatRequest.messages of type string."
What happened?
We can't use Vision with
ollama_chat
, but it's working withollama
.config.yaml
I use the OpenAI sdk with proxy set to litellm
Relevant log output
Twitter / LinkedIn details
No response