search

ollama / ollama-python

Ollama Python library
https://ollama.com
MIT License
2.71k stars 223 forks source link

Unable to get ollama serve working #89

Closed harsham05 closed 3 months ago

harsham05 commented 3 months ago

I have installed Ollama on Ubuntu. I am unable to interact with it using the Ollama Python client.

$ ollama list
NAME            ID              SIZE    MODIFIED
llama2:70b      e7f6c06ffef4    38 GB   2 hours ago
llama2:latest   78e26419b446    3.8 GB  4 days ago
$ OLLAMA_HOST=127.0.0.1:7656 ollama serve

When I try to interact with ollama in Python, I get a ResponseError. Thank you in advance.

import ollama
response = ollama.chat(model='llama2', messages=[
  {
    'role': 'user',
    'content': 'Why is the sky blue?',
  },
])
print(response['message']['content'])
image
mxyng commented 3 months ago

$ OLLAMA_HOST=127.0.0.1:7656 ollama serve

Since this is a non-standard ollama port, you need to set either OLLAMA_HOST=127.0.0.1:7656 or ollama.Client(host='127.0.0.1:7656')