1.107.x Facial grouping - MP4 thumbnail misalignment for featured image

poisonnuke commented 2 months ago

The bug

after upgrading and waiting for one night, a lot of MP4 files have been identified and grouped and popped up on the end of the list. The featured picture of those unkown persons was a random part of the video-thumbnail, it seems there is an offset, most of the time like 50-200px to the right but sometimes also to the top or bottom.

the faces are identified correctly though, so its the same person on all pictures, its just that the featured picture is taken from the wrong part of the thumbnail.

The OS that Immich Server is running on

Debian Bookworm

Version of Immich Server

1.107.2

Version of Immich Mobile App

1.107

Platform with the issue

[X] Server
[ ] Web
[ ] Mobile

Your docker-compose.yml content

name: immich

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - ${EXTERNAL_PATH}:/usr/src/app/external
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    ports:
      - "127.0.0.1:3001:3001"
    depends_on:
      - redis
      - database
    restart: always 

  immich-machine-learning:
    container_name: immich_machine_learning
    # For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag.
    # Example tag: ${IMMICH_VERSION:-release}-cuda
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    # extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration
    #   file: hwaccel.ml.yml
    #   service: cpu # set to one of [armnn, cuda, openvino, openvino-wsl] for accelerated inference - use the `-wsl` version for WSL2 where applicable
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: always

  immich-folder-album-creator:
    container_name: immich_folder_album_creator
    image: salvoxia/immich-folder-album-creator:latest
    restart: unless-stopped
    environment:
      API_URL: http://immich_server:3001/api
      API_KEY: asdfasdfasdfasdfasdfasdf
      ROOT_PATH: /usr/src/app/external
      ALBUM_LEVELS: 4
      ALBUM_SEPARATOR: " - "
      CRON_EXPRESSION: "0 0 * * *"
      TZ: Europe/Berlin

  redis:
    container_name: immich_redis
    image: registry.hub.docker.com/library/redis:6.2-alpine@sha256:84882e87b54734154586e5f8abd4dce69fe7311315e2fc6d67c29614c8de2672
    restart: always

  database:
    container_name: immich_postgres
    image: registry.hub.docker.com/tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      POSTGRES_INITDB_ARGS: '--data-checksums'
    volumes:
      - ${DB_DATA_LOCATION}:/var/lib/postgresql/data
    restart: always
    command: ["postgres", "-c" ,"shared_preload_libraries=vectors.so", "-c", 'search_path="$$user", public, vectors', "-c", "logging_collector=on", "-c", "max_wal_size=2GB", "-c", "shared_buffers=512MB", "-c", "wal_compression=on"]

volumes:
  model-cache:

Your .env content

# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

# The location where your uploaded files are stored
UPLOAD_LOCATION=./library
# The location where your database files are stored
DB_DATA_LOCATION=./postgres
EXTERNAL_PATH=/media/pics
# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=release

# Connection secret for postgres. You should change it to a random password
DB_PASSWORD=ddddddd

# The values below this line do not need to be changed
###################################################################################
DB_USERNAME=postgres
DB_DATABASE_NAME=immich

Reproduction steps

upgrade to 1.107.1 and trigger facial grouping as recommended in release notes

Relevant log output

No response

Additional information

No response

mrmorganmurphy commented 2 months ago

I have the same issue. I got many results like this:

On the machine learning container I did see many errors like this appearing:

2024-07-11 15:45:32.474024974 [W:onnxruntime:, execution_frame.cc:879 VerifyOutputSizes] Expected shape from model of {1,512} does not match actual shape of {2,512} for output 683

The OS that Immich Server is running on

Debian Bookworm

Version of Immich Server

1.108.0

Version of Immich Mobile App

1.107

Platform with the issue

[X] Server
[ ] Web
[ ] Mobile

mertalev commented 2 months ago

Can you provide a sample video where you see this issue?

poisonnuke commented 2 months ago

Can you provide a sample video where you see this issue?

unfortunatey, thats not possible. First of all, all videos where from "known" people, so i merged them into existing ones, finding the culprits isnt that easy now. And all of them are personal, so isnt really an option either :(

dont you have sample videos and pictures and could start a fresh installation and make sure the video creation time is the "latest" of all files (I guess the latest file is used for feature, isnt it?). and in the video-thumbnail you need to see the person of course (was the first frame for this issue)

immich-app / immich