immich-app / immich

High performance self-hosted photo and video management solution.
https://immich.app
GNU Affero General Public License v3.0
49.94k stars 2.65k forks source link

Search in 1.95.1 only returns a single result #7313

Closed installlooc closed 8 months ago

installlooc commented 8 months ago

The bug

When attempting to search for any details, either using the simple search box or the advanced search box, only a single result is returned.

This appears to be a regression from 1.95.0, which was working fine before I updated to 1.95.1

The OS that Immich Server is running on

Ubuntu 22.04.3 (Portainer)

Version of Immich Server

v1.95.1

Version of Immich Mobile App

v1.95.0

Platform with the issue

Your docker-compose.yml content

version: "3.8"

services:
  immich-server:
    networks:
      - proxy
    container_name: immich_server
    labels:
      - "traefik.http.routers.immich.rule=Host(`redacted`)"
      - "traefik.http.routers.immich.tls=true"
      - "traefik.http.routers.immich.tls.certresolver=cfresolver"
      - "traefik.http.routers.immich.entrypoints=websecure"
      - traefik.enable=true
    image: ghcr.io/immich-app/immich-server:v1.95.1
    ports:
      - 2283:3001
    command: [ "start.sh", "immich" ]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - stack.env
    depends_on:
      - redis
      - database
    restart: always

  immich-microservices:
    container_name: immich_microservices
    image: ghcr.io/immich-app/immich-server:v1.95.1
    networks:
      - proxy
    group_add:
      - "109" # Intel Quick Sync
    devices:
      - /dev/dri/renderD128:/dev/dri/renderD128
    command: [ "start.sh", "microservices" ]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - stack.env
    depends_on:
      - redis
      - database
    restart: always

  immich-machine-learning:
    container_name: immich_machine_learning
    networks:
      - proxy
    image: ghcr.io/immich-app/immich-machine-learning:v1.95.1
    volumes:
      - model-cache:/cache
    env_file:
      - stack.env
    restart: always

  redis:
    container_name: immich_redis
    networks:
      - proxy
    image: redis:6.2-alpine
    restart: always

  database:
    container_name: immich_postgres
    networks:
      - proxy
    image: tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0
    env_file:
      - stack.env
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
    volumes:
      - pgdata:/var/lib/postgresql/data
    restart: always
volumes:
  pgdata:
  model-cache:

networks:
  proxy:
    name: proxy

Your .env content

UPLOAD_LOCATION=redacted
DB_PASSWORD=redacted
DB_HOSTNAME=immich_postgres
DB_USERNAME=postgres
DB_DATABASE_NAME=immich
REDIS_HOSTNAME=immich_redis

Reproduction steps

1. Begin a search using the advanced search button
2. Fill in only the context field
3. Get only a single result when filled in.
4. Repeat this with any combination of fields, as long as context is filled in.

Additional information

No response

mmomjian commented 8 months ago

v1.95.1 here, unable to replicate. Out of curiosity what happens on mobile? Mobile still uses the old search platform.

installlooc commented 8 months ago

Identical results on mobile as well, nothing obvious in logs for any of the containers either

alextran1502 commented 8 months ago

If you revert back to 1.95, does it work?

installlooc commented 8 months ago

Yes, reverting to 1.95.0 restores the functionality

alextran1502 commented 8 months ago

Can you tell me more about your instance, how many users do you have? How many assets do you have?

THe things we fixed in 1.95.1 is to not return archived assets by default

installlooc commented 8 months ago

Single user with around 22000 photos, both archive and trash are empty. I can still see all the images through the default timeline and through albums, just not through search.

YoussefAswad commented 8 months ago

Same issue here, the search works as expected after running Smart Search Job, but returns to one result shortly after. The metadata search works regardless.

huydinh282 commented 8 months ago

I have the same issue, running Immich from a Synology NAS. Rerunning all Smart Search helps temporarily, but after the job is done I only see 1 photo, regardless of the keyword used.

Problem persists between 1.95.0 and 1.95.1, between different clip models, machine learning hosts, and after reinstalling the containers.

alextran1502 commented 8 months ago

Can you guys try run the Smart Search job for all and see if it fixes the issue?

huydinh282 commented 8 months ago

A full (or partial) re-run restores the search, but only for a few minutes for me. Curiously, the 1 photo changes after each re-run.

mertalev commented 8 months ago

I'm unable to reproduce this issue. To see if it has any relation to the archive change, can you try setting the Archive toggle when searching? Next, can you change the isArchived text in the URL to withArchived and enter?

huydinh282 commented 8 months ago

The issue still persists for me after checking the 2 archives option (I don't have any photo archived also). For me it looks like the model results are wiped off every 4-5 minutes after finishing/pausing a smart search job, leaving only 1 photo to search for.

I created another user and upload some photos to test. Search works ok in the alt account, but when I toggle show photo from the main account in timeline, the issue kicks in again. Disabling it restores search in alt account.

Edit: I am not able to replicate the bug in my alt account (using a dummy external library load). Alternatively, adding more direct upload or more external libraries does not fix the main account

ElksInNC commented 8 months ago

Also experienced the single search result on 1.95.1. Downgraded to 1.95.0 but left database at (image: tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0) at v0.2.0 and re-ran SYNC ALL.

Results are a full return of search as expected and previously experienced.

ElksInNC commented 8 months ago

Also experienced the single search result on 1.95.1. Downgraded to 1.95.0 but left database at (image: tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0) at v0.2.0 and re-ran SYNC ALL.

Results are a full return of search as expected and previously experienced.

Take it back - I also am losing the search results after about 5 min without doing anything else. Had them (search "red car" returned many results) and now "red car" returns one result (and it is a picture of a printed circuit board without any red at all). So that is on 1.95.0 and v0.2.0 on pgvecto.

mertalev commented 8 months ago

I think this might be a pgvecto.rs bug in that case. Out of curiosity, can you try running VACUUM FULL smart_search;?

huydinh282 commented 8 months ago

I think this might be a pgvecto.rs bug in that case. Out of curiosity, can you try running VACUUM FULL smart_search;?

EDIT: This resolved the issue for me. Leaving behind the steps I took for others:

  1. Command line at the immich_postgres container (for me, i used synology container manager -> action -> Open terminal -> Create ; SSH to the NAS also works)
  2. Run "psql -U postgres". Command line should then show psql at the start
  3. Run "\c immich" to point to the immich database
  4. run "VACUUM FULL smart_search;", took 10 seconds for my 100gb photos/videos.
  5. Search is immediately restored without requiring a smart search rerun.
ElksInNC commented 8 months ago

This worked for me as well. Docker Compose:

  1. From host machine where Docker lives: sudo docker exec -it immich_postgres bash (assuming you used defaults)
  2. as above steps 2-4
installlooc commented 8 months ago

I can confirm the VACUUM FULL smart_search has restored the search in 1.95.1 for me as well

mertalev commented 8 months ago

Based on this, I was able to reproduce the issue. It happens in two cases:

  1. When you queue all assets for smart search after smart search has already run and without changing a model
  2. If you run DELETE FROM smart_search without vacuuming and re-run smart search

The index ends up still having the entries from before the table was cleared / updated, so a full table's worth of duplicates wreaks havoc on it.

The first case is easy enough to fix in code. If the second applies to you, then running VACUUM FULL smart_search and VACUUM FULL asset_faces will take care of it.