search

imagegenius / docker-immich

Monolithic (Single) Docker Container for Immich
GNU General Public License v3.0
518 stars 27 forks source link

⚠️ BREAKING CHANGE - v1.91.0 #254

Closed martabal closed 4 months ago

martabal commented 9 months ago

Hello,

Immich v1.91.0 introduces a breaking change for everyone including the users of this AIO container. As stated in the v1.90.0 release notes, Immich will use a plugin for the postgres database which is not available in the default postgres image.

To solve this issue :

Docker-compose

  postgres14:
-   image: postgres:14
+   image: tensorchord/pgvecto-rs:pg14-v0.2.0
    ports:
      - 5432:5432
    container_name: postgres14
    environment:
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: postgres
      POSTGRES_DB: immich
    volumes:
      - path_to_postgres:/var/lib/postgresql/data

Docker CLI

docker run -d \
  --name=postgres14 \
  -e POSTGRES_USER=postgres \
  -e POSTGRES_PASSWORD=postgres \
  -e POSTGRES_DB=immich \
  -v path_to_postgres:/var/lib/postgresql/data \
  -p 5432:5432 \
-  postgres:14
+  tensorchord/pgvecto-rs:pg14-v0.2.0

Unraid

Edit your postgres container and change the Repository value to tensorchord/pgvecto-rs:pg14-v0.2.0

Postgres versions

If you don't use postgres 14, you can edit the image according to your current postgres version with :

You can migrate to this new image before v1.91.0 is released

EDITS

cedlap commented 9 months ago

Hi, I think the recommended image is tensorchord/pgvecto-rs:pg14-v0.1.11, not latest which is a nightly.

martabal commented 9 months ago

Thanks, I changed it

evaanp commented 9 months ago

After changing the image i get the following error: /usr/local/bin/docker-entrypoint.sh: line 346: exec: postgres: not found

martabal commented 9 months ago

Are you using the Postgres docker mod ?

evaanp commented 9 months ago

I'm using Cosmos-Server, which is using docker-compose i guess (maybe i misunderstood your question?)

martabal commented 9 months ago

Yes, can you share the environment variables you use for Immich ? (Don't forget to hide passwords and usernames)

evaanp commented 9 months ago

Postgres container:

POSTGRES_DB: immich POSTGRES_USER: immich POSTGRES_PASSWORD: PATH: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin LANG: en_US.utf8 PG_MAJOR: 15 PG_VERSION: 15.5 PG_SHA256: DOCKER_PG_LLVM_DEPS: llvm15-dev clang15 PGDATA: /var/lib/postgresql/data GOSU_VERSION: 1.16

Immich container; DB_HOSTNAME: Immich-postgres -- Username and pw left out -- PATH: /lsiopy/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin HOME: /root LANGUAGE: en_US.UTF-8 LANG: en_US.UTF-8 TERM: xterm S6_CMD_WAIT_FOR_SERVICES_MAXTIME: 0 S6_VERBOSITY: 1 S6_STAGE2_HOOK: /docker-mods VIRTUAL_ENV: /lsiopy IMMICH_MACHINE_LEARNING_URL: http://127.0.0.1:3003 IMMICH_MEDIA_LOCATION: /photos PUBLIC_IMMICH_SERVER_URL: http://127.0.0.1:3001 MACHINE_LEARNING_CACHE_FOLDER: /config/machine-learning TRANSFORMERS_CACHE: /config/machine-learning TYPESENSE_DATA_DIR: /config/typesense TYPESENSE_API_KEY: xyz TYPESENSE_HOST: 127.0.0.1 TYPESENSE_VERSION: 0.24.1 REVERSE_GEOCODING_DUMP_DIRECTORY: /config/.reverse-geocoding-dump/ NODE_ENV: production SERVER_PORT: 8080

Note: the postgres container isn't working at all, the log is from the postgres container.

zhono commented 9 months ago

Just FYI, after changing my postgres repository as stated, immich still could not boot. Checking the postgres logs, I saw a repeated permission denied error. I had to use pgAdmin to edit my immich database user and make it a super user, now Immich boots and runs.

Getting a warning in my postgres logs that says "WARNING: you don't own a lock of type ExclusiveLock". Not sure what that is, or if it's important, but Immich seems to work, so hopefully it doesn't matter.

edit: I'm an idiot. Just checked the page with the original announcement, and they mention the super user thing. Oh well.

martabal commented 9 months ago

I don't know about cosmos, but somehow you have to change the image used for postgres

canedje commented 9 months ago

Mine Immich is broken also I changed the repository image to: tensorchord/pgvecto-rs:pg14-v0.1.11 The docker is not starting anymore: image

I did only change the repository and changed nothing else I do running an Unraid docker image image my setup. It still show database name and passwd

martabal commented 9 months ago

You changed your Immich container not the postgres container

evaanp commented 9 months ago

I fixed it by creating a new postgres container with the same volume instead of changing the image of the existing

canedje commented 9 months ago

You changed your Immich container not the postgres container

stupid me. Your right It is working again -> Thanks

HeresJonny commented 9 months ago

I just changed the repository as recommended in the initial post and now the postgres docker won't start anymore.

"Env": [ "POSTGRES_USER=postgres", "POSTGRES_DB=postgres", "TZ=Europe/Berlin", "HOST_OS=Unraid", "HOST_HOSTNAME=Tower", "HOST_CONTAINERNAME=postgresql15", "POSTGRES_PASSWORD=*", "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/lib/postgresql/15/bin", "GOSU_VERSION=1.16", "LANG=en_US.utf8", "PG_MAJOR=15", "PG_VERSION=15.5-1.pgdg120+1", "PGDATA=/var/lib/postgresql/data"

2023-12-17 14:12:17.989 CET [1] LOG: starting PostgreSQL 15.5 (Debian 15.5-1.pgdg120+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 12.2.0-14) 12.2.0, 64-bit 2023-12-17 14:12:17.989 CET [1] LOG: listening on IPv4 address "0.0.0.0", port 5432 2023-12-17 14:12:17.989 CET [1] LOG: listening on IPv6 address "::", port 5432 2023-12-17 14:12:17.991 CET [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432" 2023-12-17 14:12:17.994 CET [29] LOG: database system was shut down at 2023-12-17 14:03:31 CET 2023-12-17 14:12:17.995 CET [29] LOG: unexpected pageaddr 0/2E752000 in log segment 000000010000000000000035, offset 7675904 2023-12-17 14:12:17.995 CET [29] LOG: invalid primary checkpoint record 2023-12-17 14:12:17.995 CET [29] PANIC: could not locate a valid checkpoint record 2023-12-17 14:12:17.996 CET [1] LOG: startup process (PID 29) was terminated by signal 6: Aborted 2023-12-17 14:12:17.996 CET [1] LOG: aborting startup due to startup process failure 2023-12-17 14:12:17.996 CET [1] LOG: database system is shut down

martabal commented 9 months ago

What image do you use ?

ktm-91 commented 9 months ago

any way to fix this while keep using Postgres docker mod?

I have DOCKER_MODS = imagegenius/mods:universal-redis|imagegenius/mods:universal-postgres

martabal commented 9 months ago

No, postgres mod has been deprecated for several months now.

HeresJonny commented 9 months ago

What image do you use ?

The official image from https://hub.docker.com/_/postgres/ with tag postgres:15

martabal commented 9 months ago

Did you replace postgres:15 by tensorchord/pgvecto-rs:pg15-v0.1.11 ?

HeresJonny commented 9 months ago

I was able to get rid of the corruption by restoring the latest backup and then changing the repository. Now it's running.

blaine07 commented 9 months ago

I fixed it by creating a new postgres container with the same volume instead of changing the image of the existing

Stupid me was Sharing Postgres container with other services on Unraid so I can't just change existing postgres to new container. FML

Yeah, so on Unraid I have no idea how I could get current stuff out of existing postgres DB and migrate to a new DB to use the new one required for Immich?

martabal commented 9 months ago

If you don't want to use the custom image, you have to spin up a new postgres container with the pgvecto.rs extension, migrate your immich database with pg_dump (like a postgres major update) and import it in the new postgres container.

blaine07 commented 9 months ago

If you don't want to use the custom image, you have to spin up a new postgres container with the pgvecto.rs extension, migrate your immich database with pg_dump (like a postgres major update) and import it in the new postgres container.

So, I spun up a instance of correct Postgres14 and copied DB folder over. Have it working on New postgres and ghcr.io/imagegenius/immich:alpine-v1.90.2-ig19 BUT when I change immich container back to "immich:alpine" to get latest it all just 502's on site again. What am I doing wrong if I could say it isn't the DB migration that failed?

Tyree commented 9 months ago

Changed the postgresql container repo and then updated immich container. Worked like a charm as far as I can tell. Thanks so much, @martabal !

martabal commented 9 months ago

So, I spun up a instance of correct Postgres14 and copied DB folder over. Have it working on New postgres and ghcr.io/imagegenius/immich:alpine-v1.90.2-ig19 BUT when I change immich container back to "immich:alpine" to get latest it all just 502's on site again. What am I doing wrong if I could say it isn't the DB migration that failed?

What image are you talking about when you say correct Postgres14 ?

Have it working on New postgres and ghcr.io/imagegenius/immich:alpine-v1.90.2-ig19 BUT when I change immich container back to "immich:alpine" to get latest it all just 502's on site again.

This is normal. The v1.91 as mentionned in the title of this issue requires you to update your postgres container, if you have 502 errors when you revert to the immich:alpine image, it means you didn't correctly migrate your database. I can't help you more without your immich/postgres logs

martabal commented 9 months ago

Changed the postgresql container repo and then updated immich container. Worked like a charm as far as I can tell. Thanks so much, @martabal !

You're welcome :smile:

blaine07 commented 9 months ago

So, I spun up a instance of correct Postgres14 and copied DB folder over. Have it working on New postgres and ghcr.io/imagegenius/immich:alpine-v1.90.2-ig19 BUT when I change immich container back to "immich:alpine" to get latest it all just 502's on site again. What am I doing wrong if I could say it isn't the DB migration that failed?

What image are you talking about when you say correct Postgres14 ?

Have it working on New postgres and ghcr.io/imagegenius/immich:alpine-v1.90.2-ig19 BUT when I change immich container back to "immich:alpine" to get latest it all just 502's on site again.

This is normal. The v1.91 as mentionned in the title of this issue requires you to update your postgres container, if you have 502 errors when you revert to the immich:alpine image, it means you didn't correctly migrate your database. I can't help you more without your immich/postgres logs

TLDR; I’m dumb

Protip for other folks: this “ CREATE EXTENSION vectors;” is detrimental 🤦🏼‍♂️

Nuuki9 commented 9 months ago

Currently I run a single Postgres 14 container, which is used by a bunch of applications. I assume that if I switch that over to the tensorchord/pgvecto-rs:pg14-v0.1.11 image, then its going to break my other applications - is that right, or is the tensorchord image a drop in replacement?

Assuming its not, then I assume I would need to spin up a new tensorchord image dedicated to Immich, and migrate the database across?

Sorry if this sounds obvious or dumb, but I want to ensure I'm 100% clear before I make changes that could have widespread implications (I will have backups etc).

blaine07 commented 9 months ago

Currently I run a single Postgres 14 container, which is used by a bunch of applications. I assume that if I switch that over to the tensorchord/pgvecto-rs:pg14-v0.1.11 image, then its going to break my other applications - is that right, or is the tensorchord image a drop in replacement?

Assuming its not, then I assume I would need to spin up a new tensorchord image dedicated to Immich, and migrate the database across?

Sorry if this sounds obvious or dumb, but I want to ensure I'm 100% clear before I make changes that could have widespread implications (I will have backups etc).

No idea if that PGVECTO breaks other stuff BUT that was same situation I was in; spun up correct Postgres and migrated so that Immich has it's own Postgres so I wouldn't have to find out what changing containers would break lol

Nuuki9 commented 9 months ago

No idea if that PGVECTO breaks other stuff BUT that was same situation I was in; spun up correct Postgres and migrated so that Immich has it's own Postgres so I wouldn't have to find out what changing containers would break lol

Yeah I'm feeling like its probably not worth the risk and that a dedicated container is sensible. If I'm going to have to migrate the database anyway I may take the opportunity to switch to the official compose deployment - this AIO image has been great but if I'm going to start running dedicated containers anyway there's a lure to using the official deployment at that point.

martabal commented 9 months ago

then its going to break my other applications - is that right, or is the tensorchord image a drop in replacement?

It's a drop in replacement, it's just the postgres image with the pgvecto.rs binary installed. But I would recommend running multiple Postgres containers if other applications depend on it

Nuuki9 commented 9 months ago

It's a drop in replacement, it's just the postgres image with the pgvecto.rs binary installed. But I would recommend running multiple Postgres containers if other applications depend on it

That's good to know - thanks.

I've previously juggled the separate vs shared DBMS question, and as I'm not using compose I went shared - I have an automated backup routine that works well, I'm on a fixed version, and its been stable and efficient. Given that Immich now requires a specific image I'll have another think about whether to split it off, but if I do that I may as well switch to compose at the same time.

myusuf3 commented 8 months ago

I did the migration and db comes up just fine and I even confirmed users were moved over into new db but I am greeted with this page on login and doesn't recognize my old logins either. Anyone else run into this?

CleanShot 2023-12-22 at 17 08 40@2x

niyazaki commented 8 months ago

Hello I'm using the standard postgres14 in unRaid (https://registry.hub.docker.com/_/postgres/) but I would like to change to fix this bug (as currently the app doesn't run because it can't find the plugin). Could you tell me how I could migrate my db to a new one ? I can create a new docker container with the pg image you advised but I don't know how to migrate afterwards. Could you help me please ? @martabal

martabal commented 8 months ago

I did the migration and db comes up just fine and I even confirmed users were moved over into new db but I am greeted with this page on login and doesn't recognize my old logins either. Anyone else run into this?

I think you have the same issue as mentioned here https://github.com/immich-app/immich/issues/5907

Could you tell me how I could migrate my db to a new one ?

This not a bug, it's a breaking change in the Immich stack. Like I said in the first post, simply change the postgres image to tensorchord/pgvecto-rs:pg14-v0.1.11. If you don't want to have a custom docker image for your postgres database, simply export your Immich database, spin up the new postgres container and import your Immich database as you would for a major version upgrade of postgres.

exil0867 commented 8 months ago

I was able to migrate to pgvecto-rs but the metadata search feature no longer works. Normal keywords work fine. Is there anything i should do?

martabal commented 8 months ago

I was able to migrate to pgvecto-rs but the metadata search feature no longer works. Normal keywords work fine. Is there anything i should do?

Sorry for the late reply. With the removal of typesense, the the metadata search does no longer work. It will come back in the following months with the refactoring of the search.

martabal commented 7 months ago

Next Immich version (v1.95.0) will require pgvecto.rs v0.2.0, you'll need to change the postgres image to:

It'll be mentionned in next Immich release (don't update your image now if you're using Immich v1.94.1)

Tyree commented 7 months ago

Is pg 14 still the suggested version, or should we move to 16?

martabal commented 7 months ago

It didn't change, Postgres 14 is still the recommended version

patzobil commented 7 months ago

I can't seem to be able to get any further anymore since v1.90.0(last working config was with v1.87.0 - reverting to this version doesn't seem to work either hopefully my DB is not corrupted since all those tests)! Also running completely fresh installs with new DB doesn't work either :( Anyone can point out the direction here? I am using "image: tensorchord/pgvecto-rs:pg14-v0.1.11"

immich_server container logs:

Migration "AddCLIPEmbeddingIndex1700713994428" failed, error: called `Result::unwrap()` on an `Err` value: Os { code: 38, kind: Unsupported, message: "Function not implemented" }
/usr/src/app/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:219
            throw new QueryFailedError_1.QueryFailedError(query, parameters, err);
                  ^
QueryFailedError: called `Result::unwrap()` on an `Err` value: Os { code: 38, kind: Unsupported, message: "Function not implemented" }
    at PostgresQueryRunner.query (/usr/src/app/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:219:19)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async AddCLIPEmbeddingIndex1700713994428.up (/usr/src/app/dist/infra/migrations/1700713994428-AddCLIPEmbeddingIndex.js:9:9)
    at async MigrationExecutor.executePendingMigrations (/usr/src/app/node_modules/typeorm/migration/MigrationExecutor.js:225:17)
    at async DataSource.runMigrations (/usr/src/app/node_modules/typeorm/data-source/DataSource.js:263:35)
    at async DatabaseRepository.runMigrations (/usr/src/app/dist/infra/repositories/database.repository.js:42:9)
    at async DatabaseService.init (/usr/src/app/dist/domain/database/database.service.js:33:9)
    at async AppService.init (/usr/src/app/dist/immich/app.service.js:56:9)
    at async AppModule.onModuleInit (/usr/src/app/dist/immich/app.module.js:32:9)
    at async callModuleInitHook (/usr/src/app/node_modules/@nestjs/core/hooks/on-module-init.hook.js:51:9) {
  query: '\n' +
    '      CREATE INDEX IF NOT EXISTS clip_index ON smart_search\n' +
    '      USING vectors (embedding cosine_ops) WITH (options = $$\n' +
    '      [indexing.hnsw]\n' +
    '      m = 16\n' +
    '      ef_construction = 300\n' +
    '      $$);',
  parameters: undefined,
  driverError: error: called `Result::unwrap()` on an `Err` value: Os { code: 38, kind: Unsupported, message: "Function not implemented" }
      at /usr/src/app/node_modules/pg/lib/client.js:526:17
      at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
      at async PostgresQueryRunner.query (/usr/src/app/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:184:25)
      at async AddCLIPEmbeddingIndex1700713994428.up (/usr/src/app/dist/infra/migrations/1700713994428-AddCLIPEmbeddingIndex.js:9:9)
      at async MigrationExecutor.executePendingMigrations (/usr/src/app/node_modules/typeorm/migration/MigrationExecutor.js:225:17)
      at async DataSource.runMigrations (/usr/src/app/node_modules/typeorm/data-source/DataSource.js:263:35)
      at async DatabaseRepository.runMigrations (/usr/src/app/dist/infra/repositories/database.repository.js:42:9)
      at async DatabaseService.init (/usr/src/app/dist/domain/database/database.service.js:33:9)
      at async AppService.init (/usr/src/app/dist/immich/app.service.js:56:9)
      at async AppModule.onModuleInit (/usr/src/app/dist/immich/app.module.js:32:9) {
    length: 157,
    severity: 'ERROR',
    code: 'XX000',
    detail: undefined,
    hint: undefined,
    position: undefined,
    internalPosition: undefined,
    internalQuery: undefined,
    where: undefined,
    schema: undefined,
    table: undefined,
    column: undefined,
    dataType: undefined,
    constraint: undefined,
    file: 'mmap.rs',
    line: '33',
    routine: undefined
  },
  length: 157,
  severity: 'ERROR',
  code: 'XX000',
  detail: undefined,
  hint: undefined,
  position: undefined,
  internalPosition: undefined,
  internalQuery: undefined,
  where: undefined,
  schema: undefined,
  table: undefined,
  column: undefined,
  dataType: undefined,
  constraint: undefined,
  file: 'mmap.rs',
  line: '33',
  routine: undefined
}
Node.js v20.11.0
Migration "AddCLIPEmbeddingIndex1700713994428" failed, error: called `Result::unwrap()` on an `Err` value: Os { code: 38, kind: Unsupported, message: "Function not implemented" }
/usr/src/app/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:219
            throw new QueryFailedError_1.QueryFailedError(query, parameters, err);
                  ^
QueryFailedError: called `Result::unwrap()` on an `Err` value: Os { code: 38, kind: Unsupported, message: "Function not implemented" }
    at PostgresQueryRunner.query (/usr/src/app/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:219:19)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async AddCLIPEmbeddingIndex1700713994428.up (/usr/src/app/dist/infra/migrations/1700713994428-AddCLIPEmbeddingIndex.js:9:9)
    at async MigrationExecutor.executePendingMigrations (/usr/src/app/node_modules/typeorm/migration/MigrationExecutor.js:225:17)
    at async DataSource.runMigrations (/usr/src/app/node_modules/typeorm/data-source/DataSource.js:263:35)
    at async DatabaseRepository.runMigrations (/usr/src/app/dist/infra/repositories/database.repository.js:42:9)
    at async DatabaseService.init (/usr/src/app/dist/domain/database/database.service.js:33:9)
    at async AppService.init (/usr/src/app/dist/immich/app.service.js:56:9)
    at async AppModule.onModuleInit (/usr/src/app/dist/immich/app.module.js:32:9)
    at async callModuleInitHook (/usr/src/app/node_modules/@nestjs/core/hooks/on-module-init.hook.js:51:9) {
  query: '\n' +
    '      CREATE INDEX IF NOT EXISTS clip_index ON smart_search\n' +
    '      USING vectors (embedding cosine_ops) WITH (options = $$\n' +
    '      [indexing.hnsw]\n' +
    '      m = 16\n' +
    '      ef_construction = 300\n' +
    '      $$);',
  parameters: undefined,
  driverError: error: called `Result::unwrap()` on an `Err` value: Os { code: 38, kind: Unsupported, message: "Function not implemented" }
      at /usr/src/app/node_modules/pg/lib/client.js:526:17
      at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
      at async PostgresQueryRunner.query (/usr/src/app/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:184:25)
      at async AddCLIPEmbeddingIndex1700713994428.up (/usr/src/app/dist/infra/migrations/1700713994428-AddCLIPEmbeddingIndex.js:9:9)
      at async MigrationExecutor.executePendingMigrations (/usr/src/app/node_modules/typeorm/migration/MigrationExecutor.js:225:17)
      at async DataSource.runMigrations (/usr/src/app/node_modules/typeorm/data-source/DataSource.js:263:35)
      at async DatabaseRepository.runMigrations (/usr/src/app/dist/infra/repositories/database.repository.js:42:9)
      at async DatabaseService.init (/usr/src/app/dist/domain/database/database.service.js:33:9)
      at async AppService.init (/usr/src/app/dist/immich/app.service.js:56:9)
      at async AppModule.onModuleInit (/usr/src/app/dist/immich/app.module.js:32:9) {
    length: 157,
    severity: 'ERROR',
    code: 'XX000',
    detail: undefined,
    hint: undefined,
    position: undefined,
    internalPosition: undefined,
    internalQuery: undefined,
    where: undefined,
    schema: undefined,
    table: undefined,
    column: undefined,
    dataType: undefined,
    constraint: undefined,
    file: 'mmap.rs',
    line: '33',
    routine: undefined
  },
  length: 157,
  severity: 'ERROR',
  code: 'XX000',
  detail: undefined,
  hint: undefined,
  position: undefined,
  internalPosition: undefined,
  internalQuery: undefined,
  where: undefined,
  schema: undefined,
  table: undefined,
  column: undefined,
  dataType: undefined,
  constraint: undefined,
  file: 'mmap.rs',
  line: '33',
  routine: undefined
}
Node.js v20.11.0

immich_postgres container logs:

PostgreSQL Database directory appears to contain a database; Skipping initialization
2024-02-18 23:19:09.083 UTC [1] LOG:  starting PostgreSQL 14.10 (Debian 14.10-1.pgdg120+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 12.2.0-14) 12.2.0, 64-bit
2024-02-18 23:19:09.098 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
2024-02-18 23:19:09.098 UTC [1] LOG:  listening on IPv6 address "::", port 5432
2024-02-18 23:19:10.071 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2024-02-18 23:19:10.296 UTC [29] LOG:  database system was shut down at 2024-02-18 23:06:34 UTC
2024-02-18 23:19:10.447 UTC [1] LOG:  database system is ready to accept connections
2024-02-18 23:20:43.945 UTC [46] ERROR:  called `Result::unwrap()` on an `Err` value: Os { code: 38, kind: Unsupported, message: "Function not implemented" }
2024-02-18 23:20:43.945 UTC [46] STATEMENT:  
          CREATE INDEX IF NOT EXISTS clip_index ON smart_search
          USING vectors (embedding cosine_ops) WITH (options = $$
          [indexing.hnsw]
          m = 16
          ef_construction = 300
          $$);
2024-02-18 23:20:47.651 UTC [45] LOG:  could not receive data from client: Connection reset by peer
2024-02-18 23:20:48.609 UTC [40] ERROR:  relation "socket_io_attachments" does not exist at character 13
2024-02-18 23:20:48.609 UTC [40] STATEMENT:  DELETE FROM socket_io_attachments WHERE created_at < now() - interval '30000 milliseconds'
2024-02-18 23:21:17.097 UTC [43] ERROR:  called `Result::unwrap()` on an `Err` value: Os { code: 38, kind: Unsupported, message: "Function not implemented" }
2024-02-18 23:21:17.097 UTC [43] STATEMENT:  
          CREATE INDEX IF NOT EXISTS clip_index ON smart_search
          USING vectors (embedding cosine_ops) WITH (options = $$
          [indexing.hnsw]
          m = 16
          ef_construction = 300
          $$);
2024-02-18 23:22:08.810 UTC [53] ERROR:  called `Result::unwrap()` on an `Err` value: Os { code: 38, kind: Unsupported, message: "Function not implemented" }
2024-02-18 23:22:08.810 UTC [53] STATEMENT:  
          CREATE INDEX IF NOT EXISTS clip_index ON smart_search
          USING vectors (embedding cosine_ops) WITH (options = $$
          [indexing.hnsw]
          m = 16
          ef_construction = 300
          $$);
2024-02-18 23:22:37.628 UTC [56] ERROR:  called `Result::unwrap()` on an `Err` value: Os { code: 38, kind: Unsupported, message: "Function not implemented" }
2024-02-18 23:22:37.628 UTC [56] STATEMENT:  
          CREATE INDEX IF NOT EXISTS clip_index ON smart_search
          USING vectors (embedding cosine_ops) WITH (options = $$
          [indexing.hnsw]
          m = 16
          ef_construction = 300
          $$);
2024-02-18 23:23:25.519 UTC [61] ERROR:  called `Result::unwrap()` on an `Err` value: Os { code: 38, kind: Unsupported, message: "Function not implemented" }
2024-02-18 23:23:25.519 UTC [61] STATEMENT:  
          CREATE INDEX IF NOT EXISTS clip_index ON smart_search
          USING vectors (embedding cosine_ops) WITH (options = $$
          [indexing.hnsw]
          m = 16
          ef_construction = 300
          $$);
2024-02-18 23:23:45.600 UTC [65] ERROR:  called `Result::unwrap()` on an `Err` value: Os { code: 38, kind: Unsupported, message: "Function not implemented" }
2024-02-18 23:23:45.600 UTC [65] STATEMENT:  
          CREATE INDEX IF NOT EXISTS clip_index ON smart_search
          USING vectors (embedding cosine_ops) WITH (options = $$
          [indexing.hnsw]
          m = 16
          ef_construction = 300
          $$);
2024-02-18 23:24:31.158 UTC [70] ERROR:  called `Result::unwrap()` on an `Err` value: Os { code: 38, kind: Unsupported, message: "Function not implemented" }
2024-02-18 23:24:31.158 UTC [70] STATEMENT:  
          CREATE INDEX IF NOT EXISTS clip_index ON smart_search
          USING vectors (embedding cosine_ops) WITH (options = $$
          [indexing.hnsw]
          m = 16
          ef_construction = 300
          $$);
2024-02-18 23:24:32.694 UTC [69] LOG:  could not receive data from client: Connection reset by peer
2024-02-18 23:24:52.527 UTC [73] ERROR:  called `Result::unwrap()` on an `Err` value: Os { code: 38, kind: Unsupported, message: "Function not implemented" }
2024-02-18 23:24:52.527 UTC [73] STATEMENT:  
          CREATE INDEX IF NOT EXISTS clip_index ON smart_search
          USING vectors (embedding cosine_ops) WITH (options = $$
          [indexing.hnsw]
          m = 16
          ef_construction = 300
          $$);
2024-02-18 23:25:39.721 UTC [78] ERROR:  called `Result::unwrap()` on an `Err` value: Os { code: 38, kind: Unsupported, message: "Function not implemented" }
2024-02-18 23:25:39.721 UTC [78] STATEMENT:  
          CREATE INDEX IF NOT EXISTS clip_index ON smart_search
          USING vectors (embedding cosine_ops) WITH (options = $$
          [indexing.hnsw]
          m = 16
          ef_construction = 300
          $$);
2024-02-18 23:25:42.182 UTC [77] LOG:  could not receive data from client: Connection reset by peer
2024-02-18 23:26:06.315 UTC [81] ERROR:  called `Result::unwrap()` on an `Err` value: Os { code: 38, kind: Unsupported, message: "Function not implemented" }
2024-02-18 23:26:06.315 UTC [81] STATEMENT:  
          CREATE INDEX IF NOT EXISTS clip_index ON smart_search
          USING vectors (embedding cosine_ops) WITH (options = $$
          [indexing.hnsw]
          m = 16
          ef_construction = 300
          $$);
2024-02-18 23:26:49.641 UTC [86] ERROR:  called `Result::unwrap()` on an `Err` value: Os { code: 38, kind: Unsupported, message: "Function not implemented" }
2024-02-18 23:26:49.641 UTC [86] STATEMENT:  
          CREATE INDEX IF NOT EXISTS clip_index ON smart_search
          USING vectors (embedding cosine_ops) WITH (options = $$
          [indexing.hnsw]
          m = 16
          ef_construction = 300
          $$);
2024-02-18 23:26:52.120 UTC [85] LOG:  could not receive data from client: Connection reset by peer
2024-02-18 23:27:16.344 UTC [88] ERROR:  relation "socket_io_attachments" does not exist at character 13
2024-02-18 23:27:16.344 UTC [88] STATEMENT:  DELETE FROM socket_io_attachments WHERE created_at < now() - interval '30000 milliseconds'
2024-02-18 23:27:16.401 UTC [90] ERROR:  called `Result::unwrap()` on an `Err` value: Os { code: 38, kind: Unsupported, message: "Function not implemented" }
2024-02-18 23:27:16.401 UTC [90] STATEMENT:  
          CREATE INDEX IF NOT EXISTS clip_index ON smart_search
          USING vectors (embedding cosine_ops) WITH (options = $$
          [indexing.hnsw]
          m = 16
          ef_construction = 300
          $$);
2024-02-18 23:27:51.879 UTC [94] ERROR:  called `Result::unwrap()` on an `Err` value: Os { code: 38, kind: Unsupported, message: "Function not implemented" }
2024-02-18 23:27:51.879 UTC [94] STATEMENT:  
          CREATE INDEX IF NOT EXISTS clip_index ON smart_search
          USING vectors (embedding cosine_ops) WITH (options = $$
          [indexing.hnsw]
          m = 16
          ef_construction = 300
          $$);
patzobil commented 6 months ago

I can't seem to be able to get any further anymore since v1.90.0(last working config was with v1.87.0 - reverting to this version doesn't seem to work either hopefully my DB is not corrupted since all those tests)! Also running completely fresh installs with new DB doesn't work either :( Anyone can point out the direction here? I am using "image: tensorchord/pgvecto-rs:pg14-v0.1.11"

immich_server container logs:

Migration "AddCLIPEmbeddingIndex1700713994428" failed, error: called `Result::unwrap()` on an `Err` value: Os { code: 38, kind: Unsupported, message: "Function not implemented" }
/usr/src/app/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:219
            throw new QueryFailedError_1.QueryFailedError(query, parameters, err);
                  ^
QueryFailedError: called `Result::unwrap()` on an `Err` value: Os { code: 38, kind: Unsupported, message: "Function not implemented" }
    at PostgresQueryRunner.query (/usr/src/app/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:219:19)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async AddCLIPEmbeddingIndex1700713994428.up (/usr/src/app/dist/infra/migrations/1700713994428-AddCLIPEmbeddingIndex.js:9:9)
    at async MigrationExecutor.executePendingMigrations (/usr/src/app/node_modules/typeorm/migration/MigrationExecutor.js:225:17)
    at async DataSource.runMigrations (/usr/src/app/node_modules/typeorm/data-source/DataSource.js:263:35)
    at async DatabaseRepository.runMigrations (/usr/src/app/dist/infra/repositories/database.repository.js:42:9)
    at async DatabaseService.init (/usr/src/app/dist/domain/database/database.service.js:33:9)
    at async AppService.init (/usr/src/app/dist/immich/app.service.js:56:9)
    at async AppModule.onModuleInit (/usr/src/app/dist/immich/app.module.js:32:9)
    at async callModuleInitHook (/usr/src/app/node_modules/@nestjs/core/hooks/on-module-init.hook.js:51:9) {
  query: '\n' +
    '      CREATE INDEX IF NOT EXISTS clip_index ON smart_search\n' +
    '      USING vectors (embedding cosine_ops) WITH (options = $$\n' +
    '      [indexing.hnsw]\n' +
    '      m = 16\n' +
    '      ef_construction = 300\n' +
    '      $$);',
  parameters: undefined,
  driverError: error: called `Result::unwrap()` on an `Err` value: Os { code: 38, kind: Unsupported, message: "Function not implemented" }
      at /usr/src/app/node_modules/pg/lib/client.js:526:17
      at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
      at async PostgresQueryRunner.query (/usr/src/app/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:184:25)
      at async AddCLIPEmbeddingIndex1700713994428.up (/usr/src/app/dist/infra/migrations/1700713994428-AddCLIPEmbeddingIndex.js:9:9)
      at async MigrationExecutor.executePendingMigrations (/usr/src/app/node_modules/typeorm/migration/MigrationExecutor.js:225:17)
      at async DataSource.runMigrations (/usr/src/app/node_modules/typeorm/data-source/DataSource.js:263:35)
      at async DatabaseRepository.runMigrations (/usr/src/app/dist/infra/repositories/database.repository.js:42:9)
      at async DatabaseService.init (/usr/src/app/dist/domain/database/database.service.js:33:9)
      at async AppService.init (/usr/src/app/dist/immich/app.service.js:56:9)
      at async AppModule.onModuleInit (/usr/src/app/dist/immich/app.module.js:32:9) {
    length: 157,
    severity: 'ERROR',
    code: 'XX000',
    detail: undefined,
    hint: undefined,
    position: undefined,
    internalPosition: undefined,
    internalQuery: undefined,
    where: undefined,
    schema: undefined,
    table: undefined,
    column: undefined,
    dataType: undefined,
    constraint: undefined,
    file: 'mmap.rs',
    line: '33',
    routine: undefined
  },
  length: 157,
  severity: 'ERROR',
  code: 'XX000',
  detail: undefined,
  hint: undefined,
  position: undefined,
  internalPosition: undefined,
  internalQuery: undefined,
  where: undefined,
  schema: undefined,
  table: undefined,
  column: undefined,
  dataType: undefined,
  constraint: undefined,
  file: 'mmap.rs',
  line: '33',
  routine: undefined
}
Node.js v20.11.0
Migration "AddCLIPEmbeddingIndex1700713994428" failed, error: called `Result::unwrap()` on an `Err` value: Os { code: 38, kind: Unsupported, message: "Function not implemented" }
/usr/src/app/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:219
            throw new QueryFailedError_1.QueryFailedError(query, parameters, err);
                  ^
QueryFailedError: called `Result::unwrap()` on an `Err` value: Os { code: 38, kind: Unsupported, message: "Function not implemented" }
    at PostgresQueryRunner.query (/usr/src/app/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:219:19)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async AddCLIPEmbeddingIndex1700713994428.up (/usr/src/app/dist/infra/migrations/1700713994428-AddCLIPEmbeddingIndex.js:9:9)
    at async MigrationExecutor.executePendingMigrations (/usr/src/app/node_modules/typeorm/migration/MigrationExecutor.js:225:17)
    at async DataSource.runMigrations (/usr/src/app/node_modules/typeorm/data-source/DataSource.js:263:35)
    at async DatabaseRepository.runMigrations (/usr/src/app/dist/infra/repositories/database.repository.js:42:9)
    at async DatabaseService.init (/usr/src/app/dist/domain/database/database.service.js:33:9)
    at async AppService.init (/usr/src/app/dist/immich/app.service.js:56:9)
    at async AppModule.onModuleInit (/usr/src/app/dist/immich/app.module.js:32:9)
    at async callModuleInitHook (/usr/src/app/node_modules/@nestjs/core/hooks/on-module-init.hook.js:51:9) {
  query: '\n' +
    '      CREATE INDEX IF NOT EXISTS clip_index ON smart_search\n' +
    '      USING vectors (embedding cosine_ops) WITH (options = $$\n' +
    '      [indexing.hnsw]\n' +
    '      m = 16\n' +
    '      ef_construction = 300\n' +
    '      $$);',
  parameters: undefined,
  driverError: error: called `Result::unwrap()` on an `Err` value: Os { code: 38, kind: Unsupported, message: "Function not implemented" }
      at /usr/src/app/node_modules/pg/lib/client.js:526:17
      at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
      at async PostgresQueryRunner.query (/usr/src/app/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:184:25)
      at async AddCLIPEmbeddingIndex1700713994428.up (/usr/src/app/dist/infra/migrations/1700713994428-AddCLIPEmbeddingIndex.js:9:9)
      at async MigrationExecutor.executePendingMigrations (/usr/src/app/node_modules/typeorm/migration/MigrationExecutor.js:225:17)
      at async DataSource.runMigrations (/usr/src/app/node_modules/typeorm/data-source/DataSource.js:263:35)
      at async DatabaseRepository.runMigrations (/usr/src/app/dist/infra/repositories/database.repository.js:42:9)
      at async DatabaseService.init (/usr/src/app/dist/domain/database/database.service.js:33:9)
      at async AppService.init (/usr/src/app/dist/immich/app.service.js:56:9)
      at async AppModule.onModuleInit (/usr/src/app/dist/immich/app.module.js:32:9) {
    length: 157,
    severity: 'ERROR',
    code: 'XX000',
    detail: undefined,
    hint: undefined,
    position: undefined,
    internalPosition: undefined,
    internalQuery: undefined,
    where: undefined,
    schema: undefined,
    table: undefined,
    column: undefined,
    dataType: undefined,
    constraint: undefined,
    file: 'mmap.rs',
    line: '33',
    routine: undefined
  },
  length: 157,
  severity: 'ERROR',
  code: 'XX000',
  detail: undefined,
  hint: undefined,
  position: undefined,
  internalPosition: undefined,
  internalQuery: undefined,
  where: undefined,
  schema: undefined,
  table: undefined,
  column: undefined,
  dataType: undefined,
  constraint: undefined,
  file: 'mmap.rs',
  line: '33',
  routine: undefined
}
Node.js v20.11.0

immich_postgres container logs:

PostgreSQL Database directory appears to contain a database; Skipping initialization
2024-02-18 23:19:09.083 UTC [1] LOG:  starting PostgreSQL 14.10 (Debian 14.10-1.pgdg120+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 12.2.0-14) 12.2.0, 64-bit
2024-02-18 23:19:09.098 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
2024-02-18 23:19:09.098 UTC [1] LOG:  listening on IPv6 address "::", port 5432
2024-02-18 23:19:10.071 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2024-02-18 23:19:10.296 UTC [29] LOG:  database system was shut down at 2024-02-18 23:06:34 UTC
2024-02-18 23:19:10.447 UTC [1] LOG:  database system is ready to accept connections
2024-02-18 23:20:43.945 UTC [46] ERROR:  called `Result::unwrap()` on an `Err` value: Os { code: 38, kind: Unsupported, message: "Function not implemented" }
2024-02-18 23:20:43.945 UTC [46] STATEMENT:  
        CREATE INDEX IF NOT EXISTS clip_index ON smart_search
        USING vectors (embedding cosine_ops) WITH (options = $$
        [indexing.hnsw]
        m = 16
        ef_construction = 300
        $$);
2024-02-18 23:20:47.651 UTC [45] LOG:  could not receive data from client: Connection reset by peer
2024-02-18 23:20:48.609 UTC [40] ERROR:  relation "socket_io_attachments" does not exist at character 13
2024-02-18 23:20:48.609 UTC [40] STATEMENT:  DELETE FROM socket_io_attachments WHERE created_at < now() - interval '30000 milliseconds'
2024-02-18 23:21:17.097 UTC [43] ERROR:  called `Result::unwrap()` on an `Err` value: Os { code: 38, kind: Unsupported, message: "Function not implemented" }
2024-02-18 23:21:17.097 UTC [43] STATEMENT:  
        CREATE INDEX IF NOT EXISTS clip_index ON smart_search
        USING vectors (embedding cosine_ops) WITH (options = $$
        [indexing.hnsw]
        m = 16
        ef_construction = 300
        $$);
2024-02-18 23:22:08.810 UTC [53] ERROR:  called `Result::unwrap()` on an `Err` value: Os { code: 38, kind: Unsupported, message: "Function not implemented" }
2024-02-18 23:22:08.810 UTC [53] STATEMENT:  
        CREATE INDEX IF NOT EXISTS clip_index ON smart_search
        USING vectors (embedding cosine_ops) WITH (options = $$
        [indexing.hnsw]
        m = 16
        ef_construction = 300
        $$);
2024-02-18 23:22:37.628 UTC [56] ERROR:  called `Result::unwrap()` on an `Err` value: Os { code: 38, kind: Unsupported, message: "Function not implemented" }
2024-02-18 23:22:37.628 UTC [56] STATEMENT:  
        CREATE INDEX IF NOT EXISTS clip_index ON smart_search
        USING vectors (embedding cosine_ops) WITH (options = $$
        [indexing.hnsw]
        m = 16
        ef_construction = 300
        $$);
2024-02-18 23:23:25.519 UTC [61] ERROR:  called `Result::unwrap()` on an `Err` value: Os { code: 38, kind: Unsupported, message: "Function not implemented" }
2024-02-18 23:23:25.519 UTC [61] STATEMENT:  
        CREATE INDEX IF NOT EXISTS clip_index ON smart_search
        USING vectors (embedding cosine_ops) WITH (options = $$
        [indexing.hnsw]
        m = 16
        ef_construction = 300
        $$);
2024-02-18 23:23:45.600 UTC [65] ERROR:  called `Result::unwrap()` on an `Err` value: Os { code: 38, kind: Unsupported, message: "Function not implemented" }
2024-02-18 23:23:45.600 UTC [65] STATEMENT:  
        CREATE INDEX IF NOT EXISTS clip_index ON smart_search
        USING vectors (embedding cosine_ops) WITH (options = $$
        [indexing.hnsw]
        m = 16
        ef_construction = 300
        $$);
2024-02-18 23:24:31.158 UTC [70] ERROR:  called `Result::unwrap()` on an `Err` value: Os { code: 38, kind: Unsupported, message: "Function not implemented" }
2024-02-18 23:24:31.158 UTC [70] STATEMENT:  
        CREATE INDEX IF NOT EXISTS clip_index ON smart_search
        USING vectors (embedding cosine_ops) WITH (options = $$
        [indexing.hnsw]
        m = 16
        ef_construction = 300
        $$);
2024-02-18 23:24:32.694 UTC [69] LOG:  could not receive data from client: Connection reset by peer
2024-02-18 23:24:52.527 UTC [73] ERROR:  called `Result::unwrap()` on an `Err` value: Os { code: 38, kind: Unsupported, message: "Function not implemented" }
2024-02-18 23:24:52.527 UTC [73] STATEMENT:  
        CREATE INDEX IF NOT EXISTS clip_index ON smart_search
        USING vectors (embedding cosine_ops) WITH (options = $$
        [indexing.hnsw]
        m = 16
        ef_construction = 300
        $$);
2024-02-18 23:25:39.721 UTC [78] ERROR:  called `Result::unwrap()` on an `Err` value: Os { code: 38, kind: Unsupported, message: "Function not implemented" }
2024-02-18 23:25:39.721 UTC [78] STATEMENT:  
        CREATE INDEX IF NOT EXISTS clip_index ON smart_search
        USING vectors (embedding cosine_ops) WITH (options = $$
        [indexing.hnsw]
        m = 16
        ef_construction = 300
        $$);
2024-02-18 23:25:42.182 UTC [77] LOG:  could not receive data from client: Connection reset by peer
2024-02-18 23:26:06.315 UTC [81] ERROR:  called `Result::unwrap()` on an `Err` value: Os { code: 38, kind: Unsupported, message: "Function not implemented" }
2024-02-18 23:26:06.315 UTC [81] STATEMENT:  
        CREATE INDEX IF NOT EXISTS clip_index ON smart_search
        USING vectors (embedding cosine_ops) WITH (options = $$
        [indexing.hnsw]
        m = 16
        ef_construction = 300
        $$);
2024-02-18 23:26:49.641 UTC [86] ERROR:  called `Result::unwrap()` on an `Err` value: Os { code: 38, kind: Unsupported, message: "Function not implemented" }
2024-02-18 23:26:49.641 UTC [86] STATEMENT:  
        CREATE INDEX IF NOT EXISTS clip_index ON smart_search
        USING vectors (embedding cosine_ops) WITH (options = $$
        [indexing.hnsw]
        m = 16
        ef_construction = 300
        $$);
2024-02-18 23:26:52.120 UTC [85] LOG:  could not receive data from client: Connection reset by peer
2024-02-18 23:27:16.344 UTC [88] ERROR:  relation "socket_io_attachments" does not exist at character 13
2024-02-18 23:27:16.344 UTC [88] STATEMENT:  DELETE FROM socket_io_attachments WHERE created_at < now() - interval '30000 milliseconds'
2024-02-18 23:27:16.401 UTC [90] ERROR:  called `Result::unwrap()` on an `Err` value: Os { code: 38, kind: Unsupported, message: "Function not implemented" }
2024-02-18 23:27:16.401 UTC [90] STATEMENT:  
        CREATE INDEX IF NOT EXISTS clip_index ON smart_search
        USING vectors (embedding cosine_ops) WITH (options = $$
        [indexing.hnsw]
        m = 16
        ef_construction = 300
        $$);
2024-02-18 23:27:51.879 UTC [94] ERROR:  called `Result::unwrap()` on an `Err` value: Os { code: 38, kind: Unsupported, message: "Function not implemented" }
2024-02-18 23:27:51.879 UTC [94] STATEMENT:  
        CREATE INDEX IF NOT EXISTS clip_index ON smart_search
        USING vectors (embedding cosine_ops) WITH (options = $$
        [indexing.hnsw]
        m = 16
        ef_construction = 300
        $$);

Fixed! Very Well done team on the v1.95.1 releases, finally got my system back! :)

exil0867 commented 6 months ago

I updated to the latest version. Weird issue I noticed that the metadata search prefix shows results but they all seem were queried using the AI model. For example m:flower123.jpg will show results of any image that has a flower and the exact file wouldn't show if it's not a picture of a flower. Before the changes I was able to search for particular images using the filename.

martabal commented 6 months ago

I updated to the latest version. Weird issue I noticed that the metadata search prefix shows results but they all seem were queried using the AI model. For example m:flower123.jpg will show results of any image that has a flower and the exact file wouldn't show if it's not a picture of a flower. Before the changes I was able to search for particular images using the filename.

https://github.com/immich-app/immich/issues/7314