[immich-cuda-node] - unable to run job handler

[travisboss]

[Nest] 810  - 07/05/2023, 7:53:41 AM   ERROR [JobService] Unable to run job handler: Error: connect ECONNREFUSED 172.22.0.8:3003
[Nest] 810  - 07/05/2023, 7:53:41 AM   ERROR [JobService] Error: connect ECONNREFUSED 172.22.0.8:3003
    at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1494:16)
[Nest] 810  - 07/05/2023, 7:53:41 AM   ERROR [JobService] Object:
{
  "id": "45f31cdc-27f7-44c1-984b-7778aba21dea",
  "source": "upload"
}

Using a Nvidia P2000

Here is the full container:

version: "2.1"
networks:
  docker-services:
    external: true
services:
  immich:
    image: ghcr.io/imagegenius/immich:latest
    runtime: nvidia
    container_name: immich
    environment:
      - PUID=1000
      - PGID=100
      - TZ=America/Chicago
      - DB_HOSTNAME=postgres14
      - DB_USERNAME=postgres
      - DB_PASSWORD=postgres
      - DB_DATABASE_NAME=immich
      - REDIS_HOSTNAME=localhost
      - DISABLE_MACHINE_LEARNING=false #optional
      - IMMICH_MACHINE_LEARNING_URL=http://immich-cuda-node:3003
      - DISABLE_TYPESENSE=false #optional
      - DB_PORT=5432 #optional
      - REDIS_PORT=6379 #optional
      - CUDA_ACCELERATION=false #optional
      - DOCKER_MODS=imagegenius/mods:universal-redis
      - NVIDIA_VISIBLE_DEVICES=all
    volumes:
      - /pool/datavault/immich/config:/config
      - /pool/datavault/immich/photos:/photos
      - /pool/datavault/immich/machine-learning:/config/machine-learning #optional
    ports:
      - 9088:8080
    restart: unless-stopped
    networks:
      - docker-services

# PostgreSQL 14:
  postgres14:
    image: postgres:14
    ports:
      - 5432:5432
    container_name: postgres14
    environment:
      PUID: 1000
      PGID: 100
      TZ: America/Chicago
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: postgres
      POSTGRES_DB: immich
    volumes:
      - /pool/datavault/immich/postgresql:/var/lib/postgresql/data
    networks:
      - docker-services

# CUDA
  immich-cuda-node:
    image: ghcr.io/imagegenius/immich-cuda-node:latest
    runtime: nvidia
    container_name: immich-cuda-node
    environment:
      - PUID=1000
      - PGID=100
      - TZ=America/Chicago
      - NVIDIA_VISIBLE_DEVICES=all
    volumes:
      - /pool/datavault/immich/cuda/config:/var/lib/postgresql/data
      - /pool/datavault/immich/photos:/photos
    ports:
      - 3003:3003
    restart: unless-stopped
    networks:
      - docker-services

Everything does seem to be uploading, but I am unsure what this error is? Is the immich-cuda-node not being used?

To add, I am using nvidia-smi in a docker container, I am adding the gpu similar to how linuxserver recommends when using that container with runtime: nvidia and NVIDIA_VISIBLE_DEVICES=all.

[travisboss]

I think I see the issue on this is I have enabled ipv6 inside of my docker network and it does not like this I see a ::1:80 type connection.

[martabal]

Indeed, it looks like a network problem.