Can't reach database server

mgrimace commented 1 year ago

Describe the bug

Been running this successfully for a while, went to check the webUI. Couldn't reach it.
Checked Portainer, noticed kaizoku-redis-1 was stopped for some reason. Restarted Redis.
Logs show the following error, repeating:

{"level":50,"time":1695144764521,"pid":6169,"hostname":"5539d3650151","err":{"type":"PrismaClientInitializationError","message":"\nInvalid `prisma.manga.findMany()` invocation:\n\n\nCan't reach database server at `db`:`5432`\n\nPlease make sure your database server is running at `db`:`5432`.","stack":"Error: \nInvalid `prisma.manga.findMany()` invocation:\n\n\nCan't reach database server at `db`:`5432`\n\nPlease make sure your database server is running at `db`:`5432`.\n    at RequestHandler.handleRequestError (/app/node_modules/.pnpm/@prisma+client@4.9.0_prisma@4.9.0/node_modules/@prisma/client/runtime/index.js:31955:13)\n    at RequestHandler.handleAndLogRequestError (/app/node_modules/.pnpm/@prisma+client@4.9.0_prisma@4.9.0/node_modules/@prisma/client/runtime/index.js:31913:12)\n    at RequestHandler.request (/app/node_modules/.pnpm/@prisma+client@4.9.0_prisma@4.9.0/node_modules/@prisma/client/runtime/index.js:31908:12)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async PrismaClient._request (/app/node_modules/.pnpm/@prisma+client@4.9.0_prisma@4.9.0/node_modules/@prisma/client/runtime/index.js:32994:16)","clientVersion":"4.9.0"},"msg":"\nInvalid `prisma.manga.findMany()` invocation:\n\n\nCan't reach database server at `db`:`5432`\n\nPlease make sure your database server is running at `db`:`5432`."}

Reproduction steps

Browse to IP:Port
No webUI
restart docker containers
Check logs
docker compose up -d starts redis, and db, but doesn't start kaizoku (hangs at recreated, does not proceed to started)

Expected behavior

Access webUI as normal

Additional context

My compose:

version: '3'

volumes:
  db:
  redis:

services:
  app:
    container_name: kaizoku
    image: ghcr.io/oae/kaizoku:latest
    environment:
      - DATABASE_URL=postgresql://kaizoku:kaizoku@db:5432/kaizoku
      - KAIZOKU_PORT=3000
      - REDIS_HOST=redis
      - REDIS_PORT=6379
      - PUID=1000
      - PGID=1000
      - TZ=America/Toronto
    volumes:
      - /mnt/media/media/manga:/data
      - /opt/appdata/kaizoku/config:/config
      - /opt/appdata/kaizoku/logs:/logs
    depends_on:
      db:
        condition: service_healthy
    ports:
      - '8089:3000'
    restart: always
    security_opt:
      - apparmor:unconfined
    networks:
      - homelab
    labels:
      com.centurylinklabs.watchtower.enable: false
  redis:
    image: redis:7-alpine
    volumes:
      - redis:/data
    security_opt:
      - apparmor:unconfined
    networks:
      - homelab
    labels:
      com.centurylinklabs.watchtower.enable: false
  db:
    image: postgres:alpine
    restart: unless-stopped
    healthcheck:
      test: ['CMD-SHELL', 'pg_isready -U kaizoku']
      interval: 5s
      timeout: 5s
      retries: 5
    environment:
      - POSTGRES_USER=kaizoku
      - POSTGRES_DB=kaizoku
      - POSTGRES_PASSWORD=kaizoku
    volumes:
      - db:/var/lib/postgresql/data
    security_opt:
      - apparmor:unconfined
    networks:
      - homelab
    labels:
      com.centurylinklabs.watchtower.enable: false

networks:
  homelab:
    driver: bridge
    external: true

mgrimace commented 1 year ago

additional data, running compose up I see the following issue:

kaizoku-db-1     | 2023-09-19 18:08:50.104 UTC [1] FATAL:  database files are incompatible with server
kaizoku-db-1     | 2023-09-19 18:08:50.104 UTC [1] DETAIL:  The data directory was initialized by PostgreSQL version 15, which is not compatible with this version 16.0.
kaizoku-db-1 exited with code 1

mgrimace commented 1 year ago

Potential Solution

My database was created in v15. Without specifying a version number in my compose, the db postgres image updated to the latest version (v16), which is incompatible, and broke Kaizoku's access to the db. Updating the database from v15 to v16 appears possible, but requires some kind of database converstion/update that is beyond me. My solution was to specify a fixed v15 of postgres in my compose file. Steps below.

Steps to fix

You may first have to run docker compose up and see what version your database was created in. If it's 15 you can use the compose line below. If it's something different, just use the applicable version. The available version tags can be found here https://hub.docker.com/_/postgres

Available v15 apline tags: 15.4-alpine3.18, 15-alpine3.18, 15.4-alpine, 15-alpine, I just went with 15-alpine, I'm not sure if that's the best choice, but it works.

Then, in compose, make the following change (if applicable). Old:

  db:
    image: postgres:alpine

New

  db:
    image: postgres:15-alpine

The DB image must have updated with Watchtower before I added in the 'ignore' label. Or, it's possibly an actual issue, I'm not sure. Either way, this was my solution in case anyone else is impacted.

mgrimace commented 1 year ago

I just double checked the default compose.yaml at it does appear that it is listed as

  db:
    image: postgres:alpine

Which may cause issues for folks who update the db image and created their database on a previous version of postgres, e.g., by using watchtower or manually. Potential fix above.

wfulcher commented 1 year ago

I also had this issue. By adding the 15-alpine it does appear to work again. Though Agreed with mgrimace, this could be an issue for others that dont know about this issue and try to use the default compose.

oae / kaizoku