[BUG] Assets without thumbnails are hidden

[bt90]

The bug

I switched immichs video encoding from h264 to hevc and tried to upload a test video from my Android device. The test file seemingly finished the upload process and is present in the library folder. Immich also assigned an asset UUID to it but it is completely missing in the web UI.

The following error pops up in the browser console:

The file itself can be properly downloaded if i manually craft the URL:

https://immich.example.com/api/asset/download/1f95d6b0-0acd-410c-81a7-3aa9e2e9207e?key=

This seems to also confuse the Android client:

My biggest gripe with this issue is not that the video thumbnail generation seems to fail but that immich hides the file from the UI if there's no thumbnail. Can we at least add some kind of fallback here? Even a grey tile would be fine.

The OS that Immich Server is running on

Ubuntu 22.04

Version of Immich Server

v1.53.0

Version of Immich Mobile App

v1.53.0 build.76

Platform with the issue

• [X] Server
• [X] Web
• [X] Mobile

Your docker-compose.yml content

version: '3.7'

services:
  immich-server:
    container_name: immich_server
    image: altran1502/immich-server:release
    entrypoint: ["/bin/sh", "./start-server.sh"]
    user: 1000:1000
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
    env_file:
      - .env
    environment:
      - NODE_ENV=production
    depends_on:
      - immich-redis
      - immich-database
      - immich-typesense
    restart: always
    labels:
      - "diun.enable=true"
  immich-microservices:
    container_name: immich_microservices
    image: altran1502/immich-server:release
    entrypoint: ["/bin/sh", "./start-microservices.sh"]
    user: 1000:1000
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - ./immich-geocoding:/usr/src/app/.reverse-geocoding-dump
    env_file:
      - .env
    environment:
      - NODE_ENV=production
    depends_on:
      - immich-redis
      - immich-database
      - immich-typesense
    restart: always
    labels:
      - "diun.enable=true"
  immich-web:
    container_name: immich_web
    image: altran1502/immich-web:release
    entrypoint: ["/bin/sh", "./entrypoint.sh"]
    env_file:
      - .env
    environment:
      # Rename these values for svelte public interface
      - PUBLIC_IMMICH_SERVER_URL=${IMMICH_SERVER_URL}
    restart: always
    labels:
      - "diun.enable=true"
  immich-redis:
    container_name: immich_redis
    image: redis:6.2
    restart: always
    volumes:
      - ./immich-redisdata:/data
    labels:
      - "diun.enable=true"
  immich-database:
    container_name: immich_postgres
    image: postgres:14
    env_file:
      - .env
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      PG_DATA: /var/lib/postgresql/data
    volumes:
      - ./immich-pgdata:/var/lib/postgresql/data
    restart: always
    labels:
      - "diun.enable=true"
  immich-proxy:
    container_name: immich_proxy
    image: altran1502/immich-proxy:release
    environment:
      # Make sure these values get passed through from the env file
      - IMMICH_SERVER_URL
      - IMMICH_WEB_URL
    ports:
      - 127.0.0.1:8087:8080
      - "[::1]:8087:8080"
    logging:
      driver: none
    depends_on:
      - immich-server
    restart: always
    labels:
      - "diun.enable=true"
  immich-typesense:
    container_name: immich_typesense
    image: typesense/typesense:0.24.0
    environment:
      - TYPESENSE_API_KEY=${TYPESENSE_API_KEY}
      - TYPESENSE_DATA_DIR=/data
    logging:
      driver: none
    volumes:
      - ./immich-tsdata:/data
    restart: always
    labels:
      - "diun.enable=true"

Your .env content

###################################################################################
# Database
###################################################################################

DB_HOSTNAME=immich_postgres
DB_USERNAME=postgres
DB_PASSWORD=xxxxxxxxxxxxx
DB_DATABASE_NAME=immich

# Optional Database settings:
# DB_PORT=5432

###################################################################################
# Redis
###################################################################################

REDIS_HOSTNAME=immich_redis

# Optional Redis settings:
# REDIS_PORT=6379
# REDIS_DBINDEX=0
# REDIS_PASSWORD=
# REDIS_SOCKET=

###################################################################################
# Upload File Location
#
# This is the location where uploaded files are stored.
###################################################################################

UPLOAD_LOCATION=/mnt/data/Bilder
GEOCODING_DATA=./immich-geocoding

###################################################################################
# Typesense
###################################################################################
TYPESENSE_API_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
TYPESENSE_HOST=immich_typesense
# TYPESENSE_ENABLED=false

###################################################################################
# Log message level - [simple|verbose]
###################################################################################

LOG_LEVEL=simple

###################################################################################
# JWT SECRET
#
# This JWT_SECRET is used to sign the authentication keys for user login
# You should set it to a long randomly generated value
# You can use this command to generate one: openssl rand -base64 128
###################################################################################

JWT_SECRET=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

###################################################################################
# Reverse Geocoding
#
# Reverse geocoding is done locally which has a small impact on memory usage
# This memory usage can be altered by changing the REVERSE_GEOCODING_PRECISION variable
# This ranges from 0-3 with 3 being the most precise
# 3 - Cities > 500 population: ~200MB RAM
# 2 - Cities > 1000 population: ~150MB RAM
# 1 - Cities > 5000 population: ~80MB RAM
# 0 - Cities > 15000 population: ~40MB RAM
####################################################################################

# DISABLE_REVERSE_GEOCODING=false
# REVERSE_GEOCODING_PRECISION=3

####################################################################################
# WEB - Optional
#
# Custom message on the login page, should be written in HTML form.
# For example:
# PUBLIC_LOGIN_PAGE_MESSAGE="This is a demo instance of Immich.<br><br>Email: <i>demo@demo.de</i><br>Password: <i>demo</i>"
####################################################################################

PUBLIC_LOGIN_PAGE_MESSAGE=

####################################################################################
# Alternative Service Addresses - Optional
#
# This is an advanced feature for users who may be running their immich services on different hosts.
# It will not change which address or port that services bind to within their containers, but it will change where other services look for their peers.
# Note: immich-microservices is bound to 3002, but no references are made
####################################################################################

# IMMICH_WEB_URL=http://immich-web:3000
# IMMICH_SERVER_URL=http://immich-server:3001
IMMICH_SERVER_URL=
IMMICH_MACHINE_LEARNING_URL=false

####################################################################################
# OAuth Setting - Optional
#
# These setting will enable OAuth login for your instance of Immich
# Folow the instructions in the page https://immich.app/docs/usage/oauth to set up your OAuth provider
####################################################################################

# OAUTH_ENABLED=false
# OAUTH_ISSUER_URL=
# OAUTH_CLIENT_ID=
# OAUTH_CLIENT_SECRET=
# OAUTH_BUTTON_TEXT=Login with OAuth
# OAUTH_AUTO_REGISTER=true
# OAUTH_SCOPE="openid profile email"

Reproduction steps

1. Switch ffmpeg to hevc
2. Upload a test video

Additional information

No response

[zackpollard]

This is very odd, as the thumbnail generation should have nothing to do with the ffmpeg settings in the admin page. Could you please upload your server and microservice logs?

[bt90]

I may have been too quick to open the issue. Apparently the job to regenerate thumbnails, as noted in the v1.52.0 release notes, is still running.

But the main point of the ticket still stands. A missing or broken thumbnail shouldn't hide an asset from the UI.

[bt90]

hmm...maybe there's something wrong with the folder structure?

Just tried to upload a new video

[Nest] 1  - 04/08/2023, 10:24:14 AM     LOG [MediaService] Start encoding video 5c5a7669-c0d7-4398-b896-39856256bfee -crf 23,-preset ultrafast,-vcodec h264,-acodec aac,-movflags faststart,-vf scale=720:-2
[Nest] 1  - 04/08/2023, 10:24:25 AM     LOG [MediaService] Start Generating Video Thumbnail
[Nest] 1  - 04/08/2023, 10:24:25 AM   ERROR [MediaService] Failed to generate thumbnail for asset: 5c5a7669-c0d7-4398-b896-39856256bfee/VIDEO
Error: ffmpeg exited with code 1: upload/upload/93a0810e-2003-41cf-be5e-2dd8d5a052dc/36d03dfc-a944-40f7-8172-b6cd5f360cf8.mp4: No such file or directory

    at ChildProcess.<anonymous> (/usr/src/app/node_modules/fluent-ffmpeg/lib/processor.js:182:22)
    at ChildProcess.emit (node:events:527:28)
    at Process.ChildProcess._handle.onexit (node:internal/child_process:291:12)
[Nest] 1  - 04/08/2023, 10:24:33 AM     LOG [MediaService] Encoding success 5c5a7669-c0d7-4398-b896-39856256bfee

[bt90]

As it's the newest file, it is displayed in the Explore tab. So the fallback to a grey tile works here:

Note that the video preview works perfectly fine:

Is this possibly related to the folder structure changes introduced in v1.52.0 ?

[bt90]

To sum it up:

• upload works
• video preview generation works
• video thumbnail generation fails because the file path is wrong or the file has already been moved

[bt90]

This looks like a timing issue. The next few sample uploads all worked perfectly fine. Triggering the job to regenerate missing thumbnails fixed the broken ones.

[jrasm91]

Two separate things:

1. The get all assets endpoint does indeed filter out assets without a thumbnail.

2. There is a bug related to storage template being applied after exif extraction and that messing up the path for other jobs. You can run encode on missing to fix that specific error.

I'd think showing assets without a thumbnail is a good idea and a potential enhancement, but not the way it currently works today.

[bt90]

I changed the title to reflect the problem.

[bt90]

Related to #2142

[bt90]

Just for reference but the issue is not limited to videos. I had to generate missing thumbnails for three of my recent photos from the last week in order for them to show up in the web UI and get the synchronized indicator in the app.

[jrasm91]

I believed this was fixed/added in #2561