mobile: thumbnails load feels slow, possibly due to extra thumbnail generation step

[Atemu]

The bug

Ever since I've started using Immich, I noticed that thumbnails feel quite slow to load to a usable resolution. Setting the option to always load remote thumbnails works around this issue causing thumbnails to load reasonably quick but that has obvious issues.

This phone is a Fairphone 4 which is by no means a particularly fast phone but it certainly isn't a slow phone either. I remember that Google photos never used to have issues like this on the much slower Oneplus 5 I used to use but I don't know how it fares on the FP4 since I stopped using Google photos years ago.

For reference, the Fossify gallery has no such issues on the same phone with the same pictures. Thumbnails take a short moment to appear but stay the background colour in the short moment until they're there. It's really quite quick though, nothing I'd be annoyed about or really notice unless I looked for it.

Not so much with Immich though unfortunately.

A thing I've noticed in this regard is that Immich has another step in the process: it loads a noticeably low-resolution thumbnail first before loading the actual thumbnail.

This might honestly be just a feeling thing where the jump between placeholder to something appearing does not result in the actual usable image to appear. Instead, a super low resolution placeholder appears and it's so low res that you can't tell what's in it in the short moment it takes to load the actual thumbnail that is high resolution enough to tell what's in the picture.

All it really does is delay the actual thumbnail from appearing and I never had a use for it. I honestly think it'd be better to skip this generation step entirely.

This is how it works when forcing remote thumbnails too and that's with the much higher latencies and lower bandwidth of a wireless connection to a network-local server. This alone might be why that feels so much better.

Would it be possible to make generating this middle step thumbnail at least optional?

Another thought that crossed my mind while writing this: The local image has to be read from disk, decoded, downscaled, encoded again and written somewhere to generate the thumbnail.
This generation step obviously takes a noticeable amount of time. Wouldn't it be better to just skip it and show the image directly, in full resolution? I don't know how feasible this is but, in theory, merely decoding the image and scaling the resulting bitmap should be rather cheap as both of those tasks can be HW-accelerated via the video codec HW (mjpeg/h.265) and generic GPU 2D accel respectively. Perhaps worth checking out as that'd save us from generating thumbnails for local files entirely.
If it turns the view too sluggish but is still faster to load than generating thumbnails, Immich could perhaps show the originals until the thumbnails are generated asynchronously and seamlessly swap those in as they finish generating.

The OS that Immich Server is running on

NixOS

Version of Immich Server

v1.116.2

Version of Immich Mobile App

v1.116.2

Platform with the issue

• [ ] Server
• [ ] Web
• [X] Mobile

Your docker-compose.yml content

name: immich
services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    ports:
      - 2283:3001
    depends_on:
      - redis
      - database
    restart: 'no'
    healthcheck:
      disable: false
    logging:
      driver: json-file
  immich-machine-learning:
    container_name: immich_machine_learning
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: 'no'
    healthcheck:
      disable: false
    logging:
      driver: json-file
  redis:
    container_name: immich_redis
    image: docker.io/redis:6.2-alpine@sha256:2d1463258f2764328496376f5d965f20c6a67f66ea2b06dc42af351f75248792
    healthcheck:
      test: redis-cli ping || exit 1
    restart: 'no'
    logging:
      driver: json-file
  database:
    container_name: immich_postgres
    image: docker.io/tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      POSTGRES_INITDB_ARGS: '--data-checksums'
    volumes:
      - ${DB_DATA_LOCATION}:/var/lib/postgresql/data
    healthcheck:
      test: pg_isready --dbname='${DB_DATABASE_NAME}' --username='${DB_USERNAME}'
        || exit 1; Chksum="$$(psql --dbname='${DB_DATABASE_NAME}' --username='${DB_USERNAME}'
        --tuples-only --no-align --command='SELECT COALESCE(SUM(checksum_failures),
        0) FROM pg_stat_database')"; echo "checksum failure count is $$Chksum"; [
        "$$Chksum" = '0' ] || exit 1
      interval: 5m
      start_interval: 30s
      start_period: 5m
    command: ["postgres", "-c", "shared_preload_libraries=vectors.so", "-c", 'search_path="$$user",
        public, vectors', "-c", "logging_collector=on", "-c", "max_wal_size=2GB",
      "-c", "shared_buffers=512MB", "-c", "wal_compression=on"]
    restart: 'no'
    logging:
      driver: json-file
volumes:
  model-cache: null

Your .env content

DB_DATABASE_NAME=immich
DB_DATA_LOCATION=/var/lib/postgresql/data/
DB_HOSTNAME=immich_postgres
DB_PASSWORD=postgres
DB_USERNAME=postgres
IMMICH_VERSION=v1.116.2
REDIS_HOSTNAME=immich_redis
UPLOAD_LOCATION=/var/lib/immich/

Reproduction steps

1. Open App
2. Scroll through pictures quickly
3. Feel sluggish thumbnail loads with a low res thumbnail step in between

Relevant log output

No response

Additional information

(Not yet using the NixOS module, still plain docker-compose.)

[bo0tzz]

I think this is a dupe of #2128, right?

Immich doesn't generate thumbnails on demand, they're created ahead of time so there's no generation step slowing things down here.

[Atemu]

Hm, no that's not quite what I'm experiencing. It doesn't cause lag or anything or takes extremely long, it just takes noticeably longer to load the thumbnails in after starting to load them than a standard gallery app and has this weird in-between step of a super low res thumbnail that does, to me, does not appear to be helpful.

When exactly does the Immich app pre-generate the thumbnails?

Also surely Immich can't pre-generate the thumbnails of all images you could possibly browse ahead of time as those can range into the 10000s, so there must be a limitation on how much and when thumbnails are generated, right?

[mertalev]

Not a mobile dev, but I don't think the app persists the thumbnails it generates. There's a low-res thumbnail that it makes and switches with a high-res thumbnail, and in-memory caches that store the raw decoded data. I don't think there's an encoding step.

Looking at the code here, it seems like it's effectively reading and decoding from disk twice to make those two thumbnails. There's almost no time saved by resizing to 32, 32 vs. width, height, so the low-res thumbnail seems kind of pointless. I guess it makes better use of the in-memory cache since more thumbnails can fit, but unless the cache hit rate is high it's probably slower than just skipping the low-res thumbnail.

[Atemu]

I don't think the app persists the thumbnails it generates.

Oh interesting, I had assumed it does. Perhaps that would be another optimisation.

Although I believe Android provides a thumbnail caching mechanism of its own for local images if I'm not mistaken. That should probably just be used if possible.

Looking at the code here

Ah thank you, I had attempted to take a look around myself.

I don't think there's an encoding step.

I had a look at what I believe to be the source for thumbnailDataWithSize() and it takes the codec aswell as the quality as parameters. We're calling it with the default of jpeg encoder and set quality to 75, so I assume it's encoding to JPEG.
If we truly never cache these thumbnails anywhere, we shouldn't be encoding anything and should just scale down the bitmap of the asset. That should be much faster than encoding two JPEGs.

[mertalev]

Oh, you're right. It's encoding two JPEGs, then decoding them back...