search

immich-app / immich

High performance self-hosted photo and video management solution.
https://immich.app
GNU Affero General Public License v3.0
44.56k stars 2.17k forks source link

[BUG] Upgrade from 1.79.1 to 1.80.0 triggering multiple errors #4853

Closed classic-mcfly closed 9 months ago

classic-mcfly commented 10 months ago

The bug

Hello,

I recently upgraded from 1.79.1 to 1.80.0. After upgrading and starting the migration job for the new folder structure the job is failing, even on higher versions.

I get the following errors after starting the server and after the failed migration:

immich_postgres          | 2023-11-05 22:16:43.068 UTC [33] ERROR:  column AssetEntity__AssetEntity_exifInfo.profileDescription does not exist at character 3856
immich_postgres          | 2023-11-05 22:24:01.253 UTC [72] STATEMENT:  SELECT DISTINCT "distinctAlias"."AssetEntity_id" AS "ids_AssetEntity_id", "distinctAlias"."AssetEntity_createdAt" FROM (SELECT "AssetEntity"."id" AS "AssetEntity_id", "AssetEntity"."deviceAssetId" AS "AssetEntity_deviceAssetId", "AssetEntity"."ownerId" AS "AssetEntity_ownerId", "AssetEntity"."libraryId" AS "AssetEntity_libraryId", "AssetEntity"."deviceId" AS "AssetEntity_deviceId", "AssetEntity"."type" AS "AssetEntity_type", "AssetEntity"."originalPath" AS "AssetEntity_originalPath", "AssetEntity"."resizePath" AS "AssetEntity_resizePath", "AssetEntity"."webpPath" AS "AssetEntity_webpPath", "AssetEntity"."thumbhash" AS "AssetEntity_thumbhash", "AssetEntity"."encodedVideoPath" AS "AssetEntity_encodedVideoPath", "AssetEntity"."createdAt" AS "AssetEntity_createdAt", "AssetEntity"."updatedAt" AS "AssetEntity_updatedAt", "AssetEntity"."fileCreatedAt" AS "AssetEntity_fileCreatedAt", "AssetEntity"."fileModifiedAt" AS "AssetEntity_fileModifiedAt", "AssetEntity"."isFavorite" AS "AssetEntity_isFavorite", "AssetEntity"."isArchived" AS "AssetEntity_isArchived", "AssetEntity"."isExternal" AS "AssetEntity_isExternal", "AssetEntity"."isReadOnly" AS "AssetEntity_isReadOnly", "AssetEntity"."isOffline" AS "AssetEntity_isOffline", "AssetEntity"."checksum" AS "AssetEntity_checksum", "AssetEntity"."duration" AS "AssetEntity_duration", "AssetEntity"."isVisible" AS "AssetEntity_isVisible", "AssetEntity"."livePhotoVideoId" AS "AssetEntity_livePhotoVideoId", "AssetEntity"."originalFileName" AS "AssetEntity_originalFileName", "AssetEntity"."sidecarPath" AS "AssetEntity_sidecarPath", "AssetEntity__AssetEntity_exifInfo"."assetId" AS "AssetEntity__AssetEntity_exifInfo_assetId", "AssetEntity__AssetEntity_exifInfo"."description" AS "AssetEntity__AssetEntity_exifInfo_description", "AssetEntity__AssetEntity_exifInfo"."exifImageWidth" AS "AssetEntity__AssetEntity_exifInfo_exifImageWidth", "AssetEntity__AssetEntity_exifInfo"."exifImageHeight" AS "AssetEntity__AssetEntity_exifInfo_exifImageHeight", "AssetEntity__AssetEntity_exifInfo"."fileSizeInByte" AS "AssetEntity__AssetEntity_exifInfo_fileSizeInByte", "AssetEntity__AssetEntity_exifInfo"."orientation" AS "AssetEntity__AssetEntity_exifInfo_orientation", "AssetEntity__AssetEntity_exifInfo"."dateTimeOriginal" AS "AssetEntity__AssetEntity_exifInfo_dateTimeOriginal", "AssetEntity__AssetEntity_exifInfo"."modifyDate" AS "AssetEntity__AssetEntity_exifInfo_modifyDate", "AssetEntity__AssetEntity_exifInfo"."timeZone" AS "AssetEntity__AssetEntity_exifInfo_timeZone", "AssetEntity__AssetEntity_exifInfo"."latitude" AS "AssetEntity__AssetEntity_exifInfo_latitude", "AssetEntity__AssetEntity_exifInfo"."longitude" AS "AssetEntity__AssetEntity_exifInfo_longitude", "AssetEntity__AssetEntity_exifInfo"."projectionType" AS "AssetEntity__AssetEntity_exifInfo_projectionType", "AssetEntity__AssetEntity_exifInfo"."city" AS "AssetEntity__AssetEntity_exifInfo_city", "AssetEntity__AssetEntity_exifInfo"."livePhotoCID" AS "AssetEntity__AssetEntity_exifInfo_livePhotoCID", "AssetEntity__AssetEntity_exifInfo"."state" AS "AssetEntity__AssetEntity_exifInfo_state", "AssetEntity__AssetEntity_exifInfo"."country" AS "AssetEntity__AssetEntity_exifInfo_country", "AssetEntity__AssetEntity_exifInfo"."make" AS "AssetEntity__AssetEntity_exifInfo_make", "AssetEntity__AssetEntity_exifInfo"."model" AS "AssetEntity__AssetEntity_exifInfo_model", "AssetEntity__AssetEntity_exifInfo"."lensModel" AS "AssetEntity__AssetEntity_exifInfo_lensModel", "AssetEntity__AssetEntity_exifInfo"."fNumber" AS "AssetEntity__AssetEntity_exifInfo_fNumber", "AssetEntity__AssetEntity_exifInfo"."focalLength" AS "AssetEntity__AssetEntity_exifInfo_focalLength", "AssetEntity__AssetEntity_exifInfo"."iso" AS "AssetEntity__AssetEntity_exifInfo_iso", "AssetEntity__AssetEntity_exifInfo"."exposureTime" AS "AssetEntity__AssetEntity_exifInfo_exposureTime", "AssetEntity__AssetEntity_exifInfo"."profileDescription" AS "AssetEntity__AssetEntity_exifInfo_profileDescription", "AssetEntity__AssetEntity_exifInfo"."colorspace" AS "AssetEntity__AssetEntity_exifInfo_colorspace", "AssetEntity__AssetEntity_exifInfo"."bitsPerSample" AS "AssetEntity__AssetEntity_exifInfo_bitsPerSample", "AssetEntity__AssetEntity_exifInfo"."fps" AS "AssetEntity__AssetEntity_exifInfo_fps", "AssetEntity__AssetEntity_exifInfo"."exifTextSearchableColumn" AS "AssetEntity__AssetEntity_exifInfo_exifTextSearchableColumn", "AssetEntity__AssetEntity_smartInfo"."assetId" AS "AssetEntity__AssetEntity_smartInfo_assetId", "AssetEntity__AssetEntity_smartInfo"."tags" AS "AssetEntity__AssetEntity_smartInfo_tags", "AssetEntity__AssetEntity_smartInfo"."objects" AS "AssetEntity__AssetEntity_smartInfo_objects", "AssetEntity__AssetEntity_smartInfo"."clipEmbedding" AS "AssetEntity__AssetEntity_smartInfo_clipEmbedding", "AssetEntity__AssetEntity_tags"."id" AS "AssetEntity__AssetEntity_tags_id", "AssetEntity__AssetEntity_tags"."type" AS "AssetEntity__AssetEntity_tags_type", "AssetEntity__AssetEntity_tags"."name" AS "AssetEntity__AssetEntity_tags_name", "AssetEntity__AssetEntity_tags"."userId" AS "AssetEntity__AssetEntity_tags_userId", "AssetEntity__AssetEntity_tags"."renameTagId" AS "AssetEntity__AssetEntity_tags_renameTagId", "AssetEntity__AssetEntity_faces"."assetId" AS "AssetEntity__AssetEntity_faces_assetId", "AssetEntity__AssetEntity_faces"."personId" AS "AssetEntity__AssetEntity_faces_personId", "AssetEntity__AssetEntity_faces"."embedding" AS "AssetEntity__AssetEntity_faces_embedding", "AssetEntity__AssetEntity_faces"."imageWidth" AS "AssetEntity__AssetEntity_faces_imageWidth", "AssetEntity__AssetEntity_faces"."imageHeight" AS "AssetEntity__AssetEntity_faces_imageHeight", "AssetEntity__AssetEntity_faces"."boundingBoxX1" AS "AssetEntity__AssetEntity_faces_boundingBoxX1", "AssetEntity__AssetEntity_faces"."boundingBoxY1" AS "AssetEntity__AssetEntity_faces_boundingBoxY1", "AssetEntity__AssetEntity_faces"."boundingBoxX2" AS "AssetEntity__AssetEntity_faces_boundingBoxX2", "AssetEntity__AssetEntity_faces"."boundingBoxY2" AS "AssetEntity__AssetEntity_faces_boundingBoxY2", "8258e303a73a72cf6abb13d73fb592dde0d68280"."id" AS "8258e303a73a72cf6abb13d73fb592dde0d68280_id", "8258e303a73a72cf6abb13d73fb592dde0d68280"."createdAt" AS "8258e303a73a72cf6abb13d73fb592dde0d68280_createdAt", "8258e303a73a72cf6abb13d73fb592dde0d68280"."updatedAt" AS "8258e303a73a72cf6abb13d73fb592dde0d68280_updatedAt", "8258e303a73a72cf6abb13d73fb592dde0d68280"."ownerId" AS "8258e303a73a72cf6abb13d73fb592dde0d68280_ownerId", "8258e303a73a72cf6abb13d73fb592dde0d68280"."name" AS "8258e303a73a72cf6abb13d73fb592dde0d68280_name", "8258e303a73a72cf6abb13d73fb592dde0d68280"."birthDate" AS "8258e303a73a72cf6abb13d73fb592dde0d68280_birthDate", "8258e303a73a72cf6abb13d73fb592dde0d68280"."thumbnailPath" AS "8258e303a73a72cf6abb13d73fb592dde0d68280_thumbnailPath", "8258e303a73a72cf6abb13d73fb592dde0d68280"."faceAssetId" AS "8258e303a73a72cf6abb13d73fb592dde0d68280_faceAssetId", "8258e303a73a72cf6abb13d73fb592dde0d68280"."isHidden" AS "8258e303a73a72cf6abb13d73fb592dde0d68280_isHidden" FROM "assets" "AssetEntity" LEFT JOIN "exif" "AssetEntity__AssetEntity_exifInfo" ON "AssetEntity__AssetEntity_exifInfo"."assetId"="AssetEntity"."id"  LEFT JOIN "smart_info" "AssetEntity__AssetEntity_smartInfo" ON "AssetEntity__AssetEntity_smartInfo"."assetId"="AssetEntity"."id"  LEFT JOIN "tag_asset" "AssetEntity_AssetEntity__AssetEntity_tags" ON "AssetEntity_AssetEntity__AssetEntity_tags"."assetsId"="AssetEntity"."id" LEFT JOIN "tags" "AssetEntity__AssetEntity_tags" ON "AssetEntity__AssetEntity_tags"."id"="AssetEntity_AssetEntity__AssetEntity_tags"."tagsId"  LEFT JOIN "asset_faces" "AssetEntity__AssetEntity_faces" ON "AssetEntity__AssetEntity_faces"."assetId"="AssetEntity"."id"  LEFT JOIN "person" "8258e303a73a72cf6abb13d73fb592dde0d68280" ON "8258e303a73a72cf6abb13d73fb592dde0d68280"."id"="AssetEntity__AssetEntity_faces"."personId" WHERE ("AssetEntity"."isVisible" = $1)) "distinctAlias" ORDER BY "distinctAlias"."AssetEntity_createdAt" ASC, "AssetEntity_id" ASC LIMIT 1001
immich_microservices     | [Nest] 7  - 11/05/2023, 10:16:43 PM     LOG [NestApplication] Nest application successfully started +24ms
immich_microservices     | [Nest] 7  - 11/05/2023, 10:16:43 PM     LOG [ImmichMicroservice] Immich Microservices is listening on http://[::1]:3002 [v1.80.0] [PRODUCTION]
immich_microservices     | [Nest] 7  - 11/05/2023, 10:16:43 PM   ERROR [JobService] Unable to run job handler (search/search-index-assets): QueryFailedError: column AssetEntity__AssetEntity_exifInfo.profileDescription does not exist
immich_microservices     | [Nest] 7  - 11/05/2023, 10:16:43 PM   ERROR [JobService] QueryFailedError: column AssetEntity__AssetEntity_exifInfo.profileDescription does not exist
immich_microservices     |     at PostgresQueryRunner.query (/usr/src/app/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:211:19)
immich_microservices     |     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
immich_microservices     |     at async SelectQueryBuilder.loadRawResults (/usr/src/app/node_modules/typeorm/query-builder/SelectQueryBuilder.js:2183:25)
immich_microservices     |     at async SelectQueryBuilder.getRawMany (/usr/src/app/node_modules/typeorm/query-builder/SelectQueryBuilder.js:646:29)
immich_microservices     |     at async SelectQueryBuilder.executeEntitiesAndRawResults (/usr/src/app/node_modules/typeorm/query-builder/SelectQueryBuilder.js:1973:26)
immich_microservices     |     at async SelectQueryBuilder.getRawAndEntities (/usr/src/app/node_modules/typeorm/query-builder/SelectQueryBuilder.js:684:29)
immich_microservices     |     at async SelectQueryBuilder.getMany (/usr/src/app/node_modules/typeorm/query-builder/SelectQueryBuilder.js:750:25)
immich_microservices     |     at async paginate (/usr/src/app/dist/infra/utils/pagination.util.js:5:19)
immich_microservices     | [Nest] 7  - 11/05/2023, 10:16:43 PM   ERROR [JobService] Object:
immich_microservices     | {}

and

immich_postgres          | 2023-11-05 22:23:49.506 UTC [66] LOG:  could not send data to client: Connection reset by peer
immich_postgres          | 2023-11-05 22:23:49.506 UTC [66] STATEMENT:  SELECT "AssetFaceEntity"."assetId" AS "AssetFaceEntity_assetId", "AssetFaceEntity"."personId" AS "AssetFaceEntity_personId", "AssetFaceEntity"."embedding" AS "AssetFaceEntity_embedding", "AssetFaceEntity"."imageWidth" AS "AssetFaceEntity_imageWidth", "AssetFaceEntity"."imageHeight" AS "AssetFaceEntity_imageHeight", "AssetFaceEntity"."boundingBoxX1" AS "AssetFaceEntity_boundingBoxX1", "AssetFaceEntity"."boundingBoxY1" AS "AssetFaceEntity_boundingBoxY1", "AssetFaceEntity"."boundingBoxX2" AS "AssetFaceEntity_boundingBoxX2", "AssetFaceEntity"."boundingBoxY2" AS "AssetFaceEntity_boundingBoxY2", "AssetFaceEntity__AssetFaceEntity_asset"."id" AS "AssetFaceEntity__AssetFaceEntity_asset_id", "AssetFaceEntity__AssetFaceEntity_asset"."deviceAssetId" AS "AssetFaceEntity__AssetFaceEntity_asset_deviceAssetId", "AssetFaceEntity__AssetFaceEntity_asset"."ownerId" AS "AssetFaceEntity__AssetFaceEntity_asset_ownerId", "AssetFaceEntity__AssetFaceEntity_asset"."libraryId" AS "AssetFaceEntity__AssetFaceEntity_asset_libraryId", "AssetFaceEntity__AssetFaceEntity_asset"."deviceId" AS "AssetFaceEntity__AssetFaceEntity_asset_deviceId", "AssetFaceEntity__AssetFaceEntity_asset"."type" AS "AssetFaceEntity__AssetFaceEntity_asset_type", "AssetFaceEntity__AssetFaceEntity_asset"."originalPath" AS "AssetFaceEntity__AssetFaceEntity_asset_originalPath", "AssetFaceEntity__AssetFaceEntity_asset"."resizePath" AS "AssetFaceEntity__AssetFaceEntity_asset_resizePath", "AssetFaceEntity__AssetFaceEntity_asset"."webpPath" AS "AssetFaceEntity__AssetFaceEntity_asset_webpPath", "AssetFaceEntity__AssetFaceEntity_asset"."thumbhash" AS "AssetFaceEntity__AssetFaceEntity_asset_thumbhash", "AssetFaceEntity__AssetFaceEntity_asset"."encodedVideoPath" AS "AssetFaceEntity__AssetFaceEntity_asset_encodedVideoPath", "AssetFaceEntity__AssetFaceEntity_asset"."createdAt" AS "AssetFaceEntity__AssetFaceEntity_asset_createdAt", "AssetFaceEntity__AssetFaceEntity_asset"."updatedAt" AS "AssetFaceEntity__AssetFaceEntity_asset_updatedAt", "AssetFaceEntity__AssetFaceEntity_asset"."fileCreatedAt" AS "AssetFaceEntity__AssetFaceEntity_asset_fileCreatedAt", "AssetFaceEntity__AssetFaceEntity_asset"."fileModifiedAt" AS "AssetFaceEntity__AssetFaceEntity_asset_fileModifiedAt", "AssetFaceEntity__AssetFaceEntity_asset"."isFavorite" AS "AssetFaceEntity__AssetFaceEntity_asset_isFavorite", "AssetFaceEntity__AssetFaceEntity_asset"."isArchived" AS "AssetFaceEntity__AssetFaceEntity_asset_isArchived", "AssetFaceEntity__AssetFaceEntity_asset"."isExternal" AS "AssetFaceEntity__AssetFaceEntity_asset_isExternal", "AssetFaceEntity__AssetFaceEntity_asset"."isReadOnly" AS "AssetFaceEntity__AssetFaceEntity_asset_isReadOnly", "AssetFaceEntity__AssetFaceEntity_asset"."isOffline" AS "AssetFaceEntity__AssetFaceEntity_asset_isOffline", "AssetFaceEntity__AssetFaceEntity_asset"."checksum" AS "AssetFaceEntity__AssetFaceEntity_asset_checksum", "AssetFaceEntity__AssetFaceEntity_asset"."duration" AS "AssetFaceEntity__AssetFaceEntity_asset_duration", "AssetFaceEntity__AssetFaceEntity_asset"."isVisible" AS "AssetFaceEntity__AssetFaceEntity_asset_isVisible", "AssetFaceEntity__AssetFaceEntity_asset"."livePhotoVideoId" AS "AssetFaceEntity__AssetFaceEntity_asset_livePhotoVideoId", "AssetFaceEntity__AssetFaceEntity_asset"."originalFileName" AS "AssetFaceEntity__AssetFaceEntity_asset_originalFileName", "AssetFaceEntity__AssetFaceEntity_asset"."sidecarPath" AS "AssetFaceEntity__AssetFaceEntity_asset_sidecarPath" FROM "asset_faces" "AssetFaceEntity" LEFT JOIN "assets" "AssetFaceEntity__AssetFaceEntity_asset" ON "AssetFaceEntity__AssetFaceEntity_asset"."id"="AssetFaceEntity"."assetId"
immich_postgres          | 2023-11-05 22:23:49.506 UTC [66] FATAL:  connection to client lost

and

immich_microservices     | <--- Last few GCs --->
immich_microservices     | 
immich_microservices     | [7:0x4d4a2840000]   115566 ms: Scavenge 2046.5 (2084.0) -> 2045.5 (2084.3) MB, 2.3 / 0.0 ms  (average mu = 0.313, current mu = 0.424) allocation failure; 
immich_microservices     | 
immich_microservices     | 
immich_microservices     | <--- JS stacktrace --->
immich_microservices     | 
immich_microservices     | FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
immich_microservices     |  1: 0xb83f50 node::Abort() [immich_microservices]
immich_microservices     |  2: 0xa94834  [immich_microservices]
immich_microservices     |  3: 0xd647c0 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [immich_microservices]
immich_microservices     |  4: 0xd64b67 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [immich_microservices]
immich_microservices     |  5: 0xf42265  [immich_microservices]
immich_microservices     |  6: 0xf43168 v8::internal::Heap::RecomputeLimits(v8::internal::GarbageCollector) [immich_microservices]
immich_microservices     |  7: 0xf53673  [immich_microservices]
immich_microservices     |  8: 0xf544e8 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [immich_microservices]
immich_microservices     |  9: 0xf2ee4e v8::internal::HeapAllocator::AllocateRawWithLightRetrySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [immich_microservices]
immich_microservices     | 10: 0xf30217 v8::internal::HeapAllocator::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [immich_microservices]
immich_microservices     | 11: 0xf113ea v8::internal::Factory::NewFillerObject(int, v8::internal::AllocationAlignment, v8::internal::AllocationType, v8::internal::AllocationOrigin) [immich_microservices]
immich_microservices     | 12: 0x12d674f v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [immich_microservices]
immich_microservices     | 13: 0x17035b9  [immich_microservices]

These errors seem to have nothing to do with the migration because they appear even without triggering the migration job. But they seem to be caused by the version upgrade. While getting the errors the server is fully reachable from a browser. No assets are listed on the main page and there are three messages with "Internal server error (Immich Server Error)".

I could add the missing column to the database but I'm not sure if this will really fix the errors because the column will be empty. A rollback to v1.79.1 is working and triggering no error messages

The OS that Immich Server is running on

Alpine Linux 3.18.4

Version of Immich Server

1.80.0

Version of Immich Mobile App

1.80.0

Platform with the issue

Your docker-compose.yml content

version: "3.8"

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:v1.80.0
    command: ["start.sh", "immich"]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
    env_file:
      - .env
    depends_on:
      - redis
      - database
      - typesense
    restart: always

  immich-microservices:
    container_name: immich_microservices
    image: ghcr.io/immich-app/immich-server:v1.80.0
    command: ["start.sh", "microservices"]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
    env_file:
      - .env
    depends_on:
      - redis
      - database
      - typesense
    restart: always

  immich-machine-learning:
    container_name: immich_machine_learning
    image: ghcr.io/immich-app/immich-machine-learning:v1.80.0
    volumes:
      # - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - model-cache:/cache
    env_file:
      - .env
    restart: always

  immich-web:
    container_name: immich_web
    image: ghcr.io/immich-app/immich-web:v1.80.0
    env_file:
      - .env
    restart: always

  typesense:
    container_name: immich_typesense
    image: typesense/typesense:0.24.1
    environment:
      - TYPESENSE_API_KEY=${TYPESENSE_API_KEY}
      - TYPESENSE_DATA_DIR=/data
    logging:
      driver: none
    volumes:
      - tsdata:/data
    restart: always

  redis:
    container_name: immich_redis
    image: redis:6.2
    restart: always

  database:
    container_name: immich_postgres
    image: postgres:14-alpine
    env_file:
      - .env
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      PG_DATA: /var/lib/postgresql/data
    volumes:
      - /immich-app/database/data:/var/lib/postgresql/data
    restart: always

  immich-proxy:
    container_name: immich_proxy
    image: ghcr.io/immich-app/immich-proxy:v1.80.0
    environment:
      # Make sure these values get passed through from the env file
      - IMMICH_SERVER_URL
      - IMMICH_WEB_URL
    ports:
      - 2283:8080
    logging:
      driver: none
    depends_on:
      - immich-server
    restart: always

  backup:
    container_name: immich_backup
    image: prodrigestivill/postgres-backup-local
    env_file:
      - .env
    environment:
      POSTGRES_HOST: database
      POSTGRES_DB: ${DB_DATABASE_NAME}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      SCHEDULE: "@daily"
      BACKUP_NUM_KEEP: 7
      BACKUP_DIR: /db_dumps
    volumes:
      - ./db_dumps:/db_dumps
    depends_on:
      - database

volumes:
  model-cache:
  tsdata:

Your .env content

###################################################################################
# Database
###################################################################################

DB_HOSTNAME=immich_postgres
DB_USERNAME=postgres
DB_PASSWORD=xxxxxxxxxxxx
DB_DATABASE_NAME=immich

# Optional Database settings:
# DB_PORT=5432

###################################################################################
# Redis
###################################################################################

REDIS_HOSTNAME=immich_redis

# REDIS_URL will be used to pass custom options to ioredis.
# Example for Sentinel
# {"sentinels":[{"host":"redis-sentinel-node-0","port":26379},{"host":"redis-sentinel-node-1","port":26379},{"host":"redis-sentinel-node-2","port":26379}],"name":"redis-sentinel"}
# REDIS_URL=ioredis://eyJzZW50aW5lbHMiOlt7Imhvc3QiOiJyZWRpcy1zZW50aW5lbDEiLCJwb3J0IjoyNjM3OX0seyJob3N0IjoicmVkaXMtc2VudGluZWwyIiwicG9ydCI6MjYzNzl9XSwibmFtZSI6Im15bWFzdGVyIn0=

# Optional Redis settings:

# Note: these parameters are not automatically passed to the Redis Container
# to do so, please edit the docker-compose.yml file as well. Redis is not configured
# via environment variables, only redis.conf or the command line

# REDIS_PORT=6379
# REDIS_DBINDEX=0
# REDIS_USERNAME=
# REDIS_PASSWORD=
# REDIS_SOCKET=

###################################################################################
# Upload File Location
#
# This is the location where uploaded files are stored.
###################################################################################

UPLOAD_LOCATION=/mnt/photos

###################################################################################
# Typesense
###################################################################################
TYPESENSE_API_KEY=xxxxxxxxxxxxxx
# TYPESENSE_ENABLED=false
# TYPESENSE_URL uses base64 encoding for the nodes json.
# Example JSON that was used:
# [
#      { 'host': 'typesense-1.example.net', 'port': '443', 'protocol': 'https' },
#      { 'host': 'typesense-2.example.net', 'port': '443', 'protocol': 'https' },
#      { 'host': 'typesense-3.example.net', 'port': '443', 'protocol': 'https' },
#  ]
# TYPESENSE_URL=ha://WwogICAgeyAnaG9zdCc6ICd0eXBlc2Vuc2UtMS5leGFtcGxlLm5ldCcsICdwb3J0JzogJzQ0MycsICdwcm90b2NvbCc6ICdodHRwcycgfSwKICAgIHsgJ2hvc3QnOiAndHlwZXNlbnNlLTIuZXhhbXBsZS5uZXQnLCAncG9ydCc6ICc0NDMnLCAncHJvdG9jb2wnOiAnaHR0cHMnIH0sCiAgICB7ICdob3N0JzogJ3R5cGVzZW5zZS0zLmV4YW1wbGUubmV0JywgJ3BvcnQnOiAnNDQzJywgJ3Byb3RvY29sJzogJ2h0dHBzJyB9LApd

###################################################################################
# Reverse Geocoding
#
# Reverse geocoding is done locally which has a small impact on memory usage
# This memory usage can be altered by changing the REVERSE_GEOCODING_PRECISION variable
# This ranges from 0-3 with 3 being the most precise
# 3 - Cities > 500 population: ~200MB RAM
# 2 - Cities > 1000 population: ~150MB RAM
# 1 - Cities > 5000 population: ~80MB RAM
# 0 - Cities > 15000 population: ~40MB RAM
####################################################################################

# DISABLE_REVERSE_GEOCODING=false
# REVERSE_GEOCODING_PRECISION=3

####################################################################################
# WEB - Optional
#
# Custom message on the login page, should be written in HTML form.
# For example:
# PUBLIC_LOGIN_PAGE_MESSAGE="This is a demo instance of Immich.<br><br>Email: <i>demo@demo.de</i><br>Password: <i>demo</i>"
####################################################################################

PUBLIC_LOGIN_PAGE_MESSAGE=

####################################################################################
# Alternative Service Addresses - Optional
#
# This is an advanced feature for users who may be running their immich services on different hosts.
# It will not change which address or port that services bind to within their containers, but it will change where other services look for their peers.
# Note: immich-microservices is bound to 3002, but no references are made
####################################################################################

IMMICH_WEB_URL=http://immich-web:3000
IMMICH_SERVER_URL=http://immich-server:3001
IMMICH_MACHINE_LEARNING_URL=http://immich-machine-learning:3003

####################################################################################
# Alternative API's External Address - Optional
#
# This is an advanced feature used to control the public server endpoint returned to clients during Well-known discovery.
# You should only use this if you want mobile apps to access the immich API over a custom URL. Do not include trailing slash.
# NOTE: At this time, the web app will not be affected by this setting and will continue to use the relative path: /api
# Examples: http://localhost:3001, http://immich-api.example.com, etc
####################################################################################

#IMMICH_API_URL_EXTERNAL=http://localhost:3001

Reproduction steps

1. `docker-compose up` to start the services and follow the logs
2. Start migration job

Additional information

The server has 16GB memory.

alextran1502 commented 10 months ago

Can you help confirm 1.80 or 1.84?

classic-mcfly commented 10 months ago

These errors occurred first on v1.80.0 but I have tested this on all versions after v1.80.0 including v1.84.0 which showed the same errors.

classic-mcfly commented 10 months ago

I just noticed that the immich CLI also exits with an error which seems to be related to the database columns:

response: {
      status: 400,
      statusText: 'Bad Request',
      headers: [Object],
      config: [Object],
      request: [ClientRequest],
      data: [Object]
    },
    isAxiosError: true,
    toJSON: [Function: toJSON]
  },
  response: {
    message: 'Error uploading file',
    error: 'QueryFailedError: null value in column "localDateTime" of relation "assets" violates not-null co
nstraint',
    statusCode: 400
  }
}
jrasm91 commented 9 months ago

Were you able to resolve the issue?

zackpollard commented 9 months ago

In the next release we have completely reworked how reverse geocoding will function. We are replacing the old library with our own implementation that uses postgres for doing the reverse geocoding. This should result in much lower memory usage overall, and also should eliminate the microservices container running out of memory. We believe this was happening due to some issue with the logic in the old geocoding implementation which was not written by us. If you still continue to experience these issues after the next release (1.89.0) please open a new issue describing the problem. Cheers!

vale46n1 commented 8 months ago

Dears I still have this problem:

from Immich side: [Nest] 1158 - 12/17/2023, 11:40:27 AM ERROR [ExceptionHandler] relation "asset_faces" does not exist QueryFailedError: relation "asset_faces" does not exist at PostgresQueryRunner.query (/app/immich/server/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:211:19) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async UsePgVectors1700713871511.up (/app/immich/server/dist/infra/migrations/1700713871511-UsePgVectors.js:9:30) at async MigrationExecutor.executePendingMigrations (/app/immich/server/node_modules/typeorm/migration/MigrationExecutor.js:225:17) at async DataSource.runMigrations (/app/immich/server/node_modules/typeorm/data-source/DataSource.js:260:35) at async DataSource.initialize (/app/immich/server/node_modules/typeorm/data-source/DataSource.js:148:17)

from DB side: 2023-12-17 11:42:15.098 CET [162] ERROR: relation "asset_faces" does not exist at character 72 2023-12-17 11:42:15.098 CET [162] STATEMENT:
SELECT CARDINALITY(embedding::real[]) as dimsize FROM asset_faces LIMIT 1

How to solve it? Thanks