[BUG] Unable to load web UI - logs show connect timeout redis

purandarrao commented 1 year ago

The bug

This is a fresh setup on ubuntu.

Below are the logs:

immich_microservices     | Error: connect ETIMEDOUT
immich_microservices     |     at Socket.<anonymous> (/usr/src/app/node_modules/ioredis/built/Redis.js:170:41)
immich_microservices     |     at Object.onceWrapper (node:events:628:28)
immich_microservices     |     at Socket.emit (node:events:514:28)
immich_microservices     |     at Socket._onTimeout (node:net:571:8)
immich_microservices     |     at listOnTimeout (node:internal/timers:569:17)
immich_microservices     |     at process.processTimers (node:internal/timers:512:7) {
immich_microservices     |   errorno: 'ETIMEDOUT',
immich_microservices     |   code: 'ETIMEDOUT',
immich_microservices     |   syscall: 'connect'
immich_microservices     | }
immich_server            | [Nest] 7  - 09/30/2023, 9:09:43 PM   ERROR [TypeOrmModule] Unable to connect to the database. Retrying (1)...
immich_server            | Error: Connection terminated due to connection timeout
immich_server            |     at Connection.<anonymous> (/usr/src/app/node_modules/pg/lib/client.js:132:73)
immich_server            |     at Object.onceWrapper (node:events:628:28)
immich_server            |     at Connection.emit (node:events:514:28)
immich_server            |     at Socket.<anonymous> (/usr/src/app/node_modules/pg/lib/connection.js:63:12)
immich_server            |     at Socket.emit (node:events:514:28)
immich_server            |     at TCP.<anonymous> (node:net:323:12)
immich_server            | Error: connect ETIMEDOUT
immich_server            |     at Socket.<anonymous> (/usr/src/app/node_modules/ioredis/built/Redis.js:170:41)
immich_server            |     at Object.onceWrapper (node:events:628:28)
immich_server            |     at Socket.emit (node:events:514:28)
immich_server            |     at Socket._onTimeout (node:net:571:8)
immich_server            |     at listOnTimeout (node:internal/timers:569:17)
immich_server            |     at process.processTimers (node:internal/timers:512:7) {
immich_server            |   errorno: 'ETIMEDOUT',
immich_server            |   code: 'ETIMEDOUT',
immich_server            |   syscall: 'connect'
immich_server            | }

I have tried restart the setup multiple times but same problem.

The OS that Immich Server is running on

Ubuntu 22.04.1 LTS

Version of Immich Server

v1.79.0

Version of Immich Mobile App

v1.79.0

Platform with the issue

[X] Server
[X] Web
[ ] Mobile

Your docker-compose.yml content

version: "3.8"

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    command: [ "start.sh", "immich" ]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
    env_file:
      - .env
    depends_on:
      - redis
      - database
      - typesense
    restart: always

  immich-microservices:
    container_name: immich_microservices
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    # extends:
    #   file: hwaccel.yml
    #   service: hwaccel
    command: [ "start.sh", "microservices" ]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
    env_file:
      - .env
    depends_on:
      - redis
      - database
      - typesense
    restart: always

  immich-machine-learning:
    container_name: immich_machine_learning
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: always

  immich-web:
    container_name: immich_web
    image: ghcr.io/immich-app/immich-web:${IMMICH_VERSION:-release}
    env_file:
      - .env
    restart: always

  typesense:
    container_name: immich_typesense
    image: typesense/typesense:0.24.1@sha256:9bcff2b829f12074426ca044b56160ca9d777a0c488303469143dd9f8259d4dd
    environment:
      - TYPESENSE_API_KEY=${TYPESENSE_API_KEY}
      - TYPESENSE_DATA_DIR=/data
      # remove this to get debug messages
      - GLOG_minloglevel=1
    volumes:
      - tsdata:/data
    restart: always

  redis:
    container_name: immich_redis
    image: redis:6.2-alpine@sha256:70a7a5b641117670beae0d80658430853896b5ef269ccf00d1827427e3263fa3
    restart: always

  database:
    container_name: immich_postgres
    image: postgres:14-alpine@sha256:28407a9961e76f2d285dc6991e8e48893503cc3836a4755bbc2d40bcc272a441
    env_file:
      - .env
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
    volumes:
      - pgdata:/var/lib/postgresql/data
    restart: always

  immich-proxy:
    container_name: immich_proxy
    image: ghcr.io/immich-app/immich-proxy:${IMMICH_VERSION:-release}
    environment:
      # Make sure these values get passed through from the env file
      - IMMICH_SERVER_URL
      - IMMICH_WEB_URL
    ports:
      - 2283:8080
    depends_on:
      - immich-server
      - immich-web
    restart: always

volumes:
  pgdata:
  model-cache:
  tsdata:

Your .env content

# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

# The location where your uploaded files are stored
UPLOAD_LOCATION=/home/kakde/external/Photos/immich

# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=release

# Connection secrets for postgres and typesense. You should change these to random passwords
TYPESENSE_API_KEY=testimmich
DB_PASSWORD=testimmich

# The values below this line do not need to be changed
###################################################################################
DB_HOSTNAME=immich_postgres
DB_USERNAME=postgres
DB_DATABASE_NAME=immich

REDIS_HOSTNAME=immich_redis

Reproduction steps

1.docker-compose up -d
2.Access UI http://localhost:2283 - doesn't load.
3.immich-server logs shows as provided in description.

Additional information

I tried steps suggested here: https://github.com/immich-app/immich/issues/2931#issuecomment-1605396092

But since this is a fresh setup I get error saying "assets" table not found when run:

select a."originalPath", count(a.id)
from assets a
group by a."originalPath"
having count(a.id) > 1;

itangqiao commented 1 year ago

same , 502

syscall: 'connect'

2023/10/09 12:09:58 | stderr | code: 'ETIMEDOUT', 2023/10/09 12:09:58 | stderr | errorno: 'ETIMEDOUT', 2023/10/09 12:09:58 | stderr | at process.processTimers (node:internal/timers:512:7) { 2023/10/09 12:09:58 | stderr | at listOnTimeout (node:internal/timers:569:17) 2023/10/09 12:09:58 | stderr | at Socket._onTimeout (node:net:571:8) 2023/10/09 12:09:58 | stderr | at Socket.emit (node:events:514:28) 2023/10/09 12:09:58 | stderr | at Object.onceWrapper (node:events:628:28) 2023/10/09 12:09:58 | stderr | at Socket. (/app/immich/server/node_modules/ioredis/built/Redis.js:170:41) 2023/10/09 12:09:58 | stderr | Error: connect ETIMEDOUT 2023/10/09 12:09:58 | stderr | } 2023/10/09 12:09:58 | stderr | syscall: 'connect' 2023/10/09 12:09:58 | stderr | code: 'ETIMEDOUT', 2023/10/09 12:09:58 | stderr | errorno: 'ETIMEDOUT', 2023/10/09 12:09:58 | stderr | at process.processTimers (node:internal/timers:512:7) { 2023/10/09 12:09:58 | stderr | at listOnTimeout (node:internal/timers:569:17) 2023/10/09 12:09:58 | stderr | at Socket._onTimeout (node:net:571:8) 2023/10/09 12:09:58 | stderr | at Socket.emit (node:events:514:28) 2023/10/09 12:09:58 | stderr | at Object.onceWrapper (node:events:628:28) 2023/10/09 12:09:58 | stderr | at Socket. (/app/immich/server/node_modules/ioredis/built/Redis.js:170:41) 2023/10/09 12:09:58 | stderr | Error: connect ETIMEDOUT 2023/10/09 12:09:58 | stderr | }

DanielEcho commented 1 year ago

same [Nest] 7 - 10/09/2023, 12:42:14 PM ERROR [TypeOrmModule] Unable to connect to the database. Retrying (9)... Error: Connection terminated due to connection timeout at Connection. (/usr/src/app/node_modules/pg/lib/client.js:132:73) at Object.onceWrapper (node:events:628:28) at Connection.emit (node:events:514:28) at Socket. (/usr/src/app/node_modules/pg/lib/connection.js:63:12) at Socket.emit (node:events:514:28) at TCP. (node:net:323:12) [Nest] 7 - 10/09/2023, 12:42:14 PM ERROR [ExceptionHandler] Connection terminated due to connection timeout Error: Connection terminated due to connection timeout at Connection. (/usr/src/app/node_modules/pg/lib/client.js:132:73) at Object.onceWrapper (node:events:628:28) at Connection.emit (node:events:514:28) at Socket. (/usr/src/app/node_modules/pg/lib/connection.js:63:12) at Socket.emit (node:events:514:28) at TCP. (node:net:323:12)

DanielEcho commented 1 year ago

i solved this problem self. try to stop your firewall.

itangqiao commented 1 year ago

i solved this problem self. try to stop your firewall.↳

You're right, bro！ I turned off the firewall and turned it on again. It works! That's Stranger Things.

purandarrao commented 1 year ago

I restarted disaled and also tried restart firewall service, but didn't work. It started to work after I finally restarted ubuntu server itself after following the steps here https://serverfault.com/questions/129086/how-to-start-stop-iptables-on-ubuntu Thanks @DanielEcho @itangqiao

f-x1-1 commented 8 months ago

Same issue here on unraid using immich docker container. After updating the docker-container.yml, immich stopped working for me, I have tried to install it muliple times, cant make it work.

Error: connect ETIMEDOUT at Socket.<anonymous> (/app/immich/server/node_modules/ioredis/built/Redis.js:170:41) at Object.onceWrapper (node:events:632:28) at Socket.emit (node:events:518:28) at Socket._onTimeout (node:net:589:8) at listOnTimeout (node:internal/timers:573:17) at process.processTimers (node:internal/timers:514:7) { errorno: 'ETIMEDOUT', code: 'ETIMEDOUT', syscall: 'connect' } [Nest] 222 - 02/27/2024, 7:26:27 PM ERROR [TypeOrmModule] Unable to connect to the database. Retrying (1)... Error: Connection terminated due to connection timeout at Connection.<anonymous> (/app/immich/server/node_modules/pg/lib/client.js:132:73) at Object.onceWrapper (node:events:632:28) at Connection.emit (node:events:518:28) at Socket.<anonymous> (/app/immich/server/node_modules/pg/lib/connection.js:63:12) at Socket.emit (node:events:518:28) at TCP.<anonymous> (node:net:337:12) [Nest] 222 - 02/27/2024, 7:26:27 PM ERROR [TypeOrmModule] Unable to connect to the database. Retrying (2)... Error: Connection terminated due to connection timeout at Connection.<anonymous> (/app/immich/server/node_modules/pg/lib/client.js:132:73) at Object.onceWrapper (node:events:632:28) at Connection.emit (node:events:518:28) at Socket.<anonymous> (/app/immich/server/node_modules/pg/lib/connection.js:63:12) at Socket.emit (node:events:518:28) at TCP.<anonymous> (node:net:337:12)

shierro commented 8 months ago

same issue here, on ubuntu 22.04, latest docker & compose

Tywele commented 6 months ago

I experience the same issue on Fedora 39 Server Edition. Trying to install Immich via Ansible.

Here are my tasks:

- name: create Immich network
  community.docker.docker_network:
    name: immich-network
    state: present

- name: deploy Immich-Redis
  community.docker.docker_container:
    name: immich-redis
    image: registry.hub.docker.com/library/redis:6.2-alpine@sha256:84882e87b54734154586e5f8abd4dce69fe7311315e2fc6d67c29614c8de2672
    restart_policy: always
    networks: 
      - name: immich-network

- name: deploy Immich-Postgres
  community.docker.docker_container:
    name: immich-postgres
    image: registry.hub.docker.com/tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0
    restart_policy: always
    volumes:
      - "{{ nvme_mount_point }}/immich/postgres:/var/lib/postgresql/data"
    env:
      POSTGRES_DB: "{{ immich_postgres_db_name }}"
      POSTGRES_USER: "{{ immich_postgres_db_user }}"
      POSTGRES_PASSWORD: "{{ immich_postgres_db_password }}"
    networks: 
      - name: immich-network

- name: deploy Immich-Machine-Learning
  community.docker.docker_container:
    name: immich-machine-learning
    image: ghcr.io/immich-app/immich-machine-learning:release
    restart_policy: always
    volumes:
      - "{{ nvme_mount_point }}/immich/model-cache:/cache"
    networks: 
      - name: immich-network
    env:
      DB_DATABASE_NAME: "{{ immich_postgres_db_name }}"
      DB_USERNAME: "{{ immich_postgres_db_user }}"
      DB_PASSWORD: "{{ immich_postgres_db_password }}"

- name: deploy Immich-Microservices
  community.docker.docker_container:
    name: immich-microservices
    image: ghcr.io/immich-app/immich-server:release
    restart_policy: always
    command: ['start.sh', 'microservices']
    volumes:
      - "{{ hdd_mount_point}}/immich/library:/usr/src/app/upload"
      - /etc/localtime:/etc/localtime
    networks: 
      - name: immich-network
    env:
      DB_DATABASE_NAME: "{{ immich_postgres_db_name }}"
      DB_USERNAME: "{{ immich_postgres_db_user }}"
      DB_PASSWORD: "{{ immich_postgres_db_password }}"

- name: deploy Immich-Server
  community.docker.docker_container:
    name: immich-server
    image: ghcr.io/immich-app/immich-server:release
    restart_policy: always
    command: ['start.sh', 'immich']
    volumes:
      - "{{ hdd_mount_point}}/immich/library:/usr/src/app/upload"
      - /etc/localtime:/etc/localtime
    ports:
      - "2283:3001"
    networks: 
      - name: immich-network
    env:
      DB_DATABASE_NAME: "{{ immich_postgres_db_name }}"
      DB_USERNAME: "{{ immich_postgres_db_user }}"
      DB_PASSWORD: "{{ immich_postgres_db_password }}"

And the corresponding variables:

immich_postgres_db_name: immich
immich_postgres_db_user: postgres
immich_postgres_db_password: postgres

lsrom commented 6 months ago

Same issue on Unraid, pulled latest docker image from ghcr.io/imagegenius/immich:latest.

Error: connect ETIMEDOUT
    at Socket.<anonymous> (/app/immich/server/node_modules/ioredis/built/Redis.js:170:41)
    at Object.onceWrapper (node:events:633:28)
    at Socket.emit (node:events:519:28)
    at Socket._onTimeout (node:net:590:8)
    at listOnTimeout (node:internal/timers:573:17)
    at process.processTimers (node:internal/timers:514:7) {
  errorno: 'ETIMEDOUT',
  code: 'ETIMEDOUT',
  syscall: 'connect'
}

notthedan commented 5 months ago

Was migrating immich to a new VM and came across this issue.

Just renamed my redis container from immich_redis to redis. Seemed to fix the issue.

kellandJuniper commented 1 month ago

Same issue on Ubuntu.

immich_server            | [Nest] 17  - 10/05/2024, 2:41:30 AM   ERROR [TypeOrmModule] Unable to connect to the database. Retrying (1)...
immich_server            | Error: Connection terminated unexpectedly
immich_server            |     at Connection.<anonymous> (/usr/src/app/node_modules/pg/lib/client.js:131:73)
immich_server            |     at Object.onceWrapper (node:events:633:28)
immich_server            |     at Connection.emit (node:events:519:28)
immich_server            |     at Socket.<anonymous> (/usr/src/app/node_modules/pg/lib/connection.js:62:12)
immich_server            |     at Socket.emit (node:events:519:28)
immich_server            |     at TCP.<anonymous> (node:net:339:12)

Owen000 commented 1 month ago

Same issue deploying with helm on k8s

harryfine commented 1 month ago

Same here. Using Synology and recommended Portainer install. All containers show running and healthy. Firewall on Synology is disabled. Auto-block on Synology is disabled. Firewall on local Linux PC disabled. Yet, when I try the URL http://192.168.1.103:2283, I get: } Error: connect ETIMEDOUT

at Socket.<anonymous> (/usr/src/app/node_modules/ioredis/built/Redis.js:170:41)

at Object.onceWrapper (node:events:633:28)

at Socket.emit (node:events:519:28)

at Socket._onTimeout (node:net:591:8)

at listOnTimeout (node:internal/timers:581:17)

at process.processTimers (node:internal/timers:519:7) {

errorno: 'ETIMEDOUT',

code: 'ETIMEDOUT',

syscall: 'connect'

}

On the browser, it shows the connection was re-set.

AndyChenSH commented 1 month ago

Same here. Using Synology and recommended Portainer install. All containers show running and healthy. Firewall on Synology is disabled. Auto-block on Synology is disabled. Firewall on local Linux PC disabled. Yet, when I try the URL http://192.168.1.103:2283, I get: } Error: connect ETIMEDOUT
at Socket.<anonymous> (/usr/src/app/node_modules/ioredis/built/Redis.js:170:41)

at Object.onceWrapper (node:events:633:28)

at Socket.emit (node:events:519:28)

at Socket._onTimeout (node:net:591:8)

at listOnTimeout (node:internal/timers:581:17)

at process.processTimers (node:internal/timers:519:7) {
errorno: 'ETIMEDOUT',

code: 'ETIMEDOUT',

syscall: 'connect'

}

On the browser, it shows the connection was re-set.

It's because the firewall blocks the communications among the immich application containers Here's how I solve this issue:

Get the Gateway and IP address from immich-app container. It could be as 172.9.0.1 / 172.9.0.5
Goto Synology Control Panel -> Security -> Firewall, edit the rule, and add a new rule as: Ports: All, Source IP: subnet 172.9.0.0 / 255.255.255.0, Action: Allow
Adjust the rule order and make sure any Deny rules will not skip this new rule.
Save and apply the change
Go back to the Container and restart immich-app

For other systems than Synology DSM, the solution concept should be the same - check the rules of host's firewall

harryfine commented 1 month ago

@AndyChenSH you were correct, I disabled the firewall and was able to log in. Now to add the entry. When you say "Get the Gateway and IP address from immich-app container. It could be as 172.9.0.1 / 172.9.0.5", which of the 4 immich containers. Here's as screenshot of my containers. I am VERY new with Docker. 2024-10-15_16-22

LightYear512 commented 3 weeks ago

你是对的，我禁用了防火墙并能够登录。现在添加条目。当您说“从 immich-app 容器中获取网关和 IP 地址。它可以是 172.9.0.1 / 172.9.0.5“，这是 4 个容器中的哪一个。这是我的容器的屏幕截图。我对 Docker 非常陌生。你可以查看 "IP Address" 列, 这里显示你的 "immich_server" 的 IP地址是 172.23.0.2, 也就是在 172.23.0.0 子网下, 所以你只需要在编辑防火墙规则时, 子网一栏填写 172.23.0.1 , 其他流程一致

immich-app / immich