Client error '404 Not Found' for url when communicating with ollama

[fidesachates]

I am getting the following 404 error when I try to chat with private gpt. When I chat with ollama directly, things work just fine.

02:14:30.002 [WARNING ] llama_index.core.chat_engine.types - Encountered exception writing response to history: Client error '404 Not Found' for url 'http://host.docker.internal:11434/api/chat

I have run through the install steps inside an ubuntu 22.04 docker container following the ollama and postgres setup. As such, I've set the api_base in the settings yaml file as http://host.docker.internal:11434.

From within the docker container, I was able to run curl http://host.docker.internal:11434/api/tags and get a successful response indicating the networking between the container and my ollama server is working. I have also another webgui in a docker container that is able to communicate with ollama perfectly fine.

This is my settings file named settings-friday.yaml:

# Using ollama and postgres for the vector, doc and index store. Ollama is also used for embeddings.
# To use install these extras:
# poetry install --extras "llms-ollama ui vector-stores-postgres embeddings-ollama storage-nodestore-postgres"
server:
  env_name: ${APP_ENV:friday}

llm:
  mode: ollama
  max_new_tokens: 512
  context_window: 3900

embedding:
  mode: ollama
  embed_dim: 768

ollama:
  llm_model: mistral
  embedding_model: nomic-embed-text
  api_base: http://host.docker.internal:11434

nodestore:
  database: postgres

vectorstore:
  database: postgres

postgres:
  host: <redacted>
  port: 5432
  database: <redacted>
  user: <redacted>
  password: <redacted>
  schema_name: privategpt

I know there is a similar issue, but the suggestion there didn't quite apply to my situation as I need to user pgvector nor did the suggestion about setting OLLAMA_HOST=host.docker.internal

[carlowisse]

I am on Fedora 39 using ollama and having the same issue.

[GIN] | 404 | 111.691µs | 127.0.0.1 | POST "/api/chat"

[INFO] httpx - HTTP Request: POST http://localhost:11434/api/chat "HTTP/1.1 404 Not Found"

[Shanorino]

I am encountering the same issue. The docker-compose file I am using is:

services:
  private-gpt:
    build:
      dockerfile: Dockerfile.external
    volumes:
      - ./local_data/:/home/worker/app/local_data
      - ./cache/tiktoken:/home/worker/app/tiktoken_cache
      - ./cache/models:/home/worker/app/models/cache
    ports:
      - 8001:8080
    environment:
      PORT: 8080
      PGPT_PROFILES: docker
      PGPT_MODE: ollama
      HUGGINGFACE_TOKEN: <token>
  ollama:
    image: ollama/ollama:latest
    volumes:
      - ./models:/root/.ollama
    expose:
      - 11434
    ports:
      - 11434:11434

Inside my "private-gpt-1" container I am able to run curl ollama:11434/api/tags

worker@370bbfe3cd67:/home/worker/app$ curl ollama:11434/api/tags
{"models":[{"name":"nomic-embed-text:latest","model":"nomic-embed-text:latest","modified_at":"2024-05-09T23:46:07.3335796Z","size":274302450,"digest":"0a109f422b47e3a30ba2b10eca18548e944e8a23073ee3f3e947efcf3c45e59f","details":{"parent_model":"","format":"gguf","family":"nomic-bert","families":["nomic-bert"],"parameter_size":"137M","quantization_level":"F16"}}]}worker@370bbfe3cd67:/home/worker/app$ 

However when I try to run the chat in UI, it seems to fail at httpx - HTTP Request: POST http://ollama:11434/api/chat "HTTP/1.1 404 Not Found

Update: I was able to fix the issue by pulling the models manually in my "ollama" container (of course the 404 error comes from the missing models):

ollama pull mistral
ollama pull nomic-embed-text