v1.107.2 - Mobile Proxy Headers Not Working (I think?)

Sammy1Am commented 1 week ago

The bug

I have Immich working fine locally, and it works through the Cloudflare Tunnel if I'm using a web browser externally. Was super excited to see #10588 merged, but after adding in the headers on the mobile app I'm not able to login to my server.

When I try to login I get a red error box that says, "Your app major version is not compatible with the server!" which is an unexpected error. Each time I attempt to login, the logs on the mobile app (even on FINEST) show only a "Pinging server with response code 200" log entry. Cloudflare doesn't report the service token has having been used, and the server doesn't have any log entries from my login attempts.

I'm a bit stuck as to how to proceed with troubleshooting the issue. Why would I get a version mismatch error? Why no error in the logs (or why does the API ping succeed)? I don't know for sure this is actually a bug, but need help figuring out what's going on.

(Probably unrelated, but the Proxy Headers UI in the mobile app has two fields: "Header name" and "Header name"-- I assume the second one is the value.)

The OS that Immich Server is running on

Immich Docker - release

Version of Immich Server

v1.107.2

Version of Immich Mobile App

v1.107.1

Platform with the issue

[ ] Server
[ ] Web
[X] Mobile

Your docker-compose.yml content

name: immich

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    # command: ['start.sh', 'immich']
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    # ports:
    #   - 2283:3001
    depends_on:
      - redis
      - database
    restart: always
    labels:
      - "traefik.enable=true"

      - "traefik.http.routers.immich.tls=true"
      - "traefik.http.routers.immich.rule=Host(`myhost.domain.com`) || Host(`myhost.internal.domain.com`)"
      - "traefik.http.routers.immich.entrypoints=websecure"
      - "traefik.http.routers.immich.service=immich"

      - "traefik.http.services.immich.loadbalancer.server.port=3001"
    networks:
      default:
      tf0:
        aliases:
          - immich-server.docc

  # immich-microservices:
  #   container_name: immich_microservices
  #   image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
  #   extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/hardware-transcoding
  #     file: hwaccel.transcoding.yml
  #     service: vaapi # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding
  #   command: ['start.sh', 'microservices']
  #   volumes:
  #     - ${UPLOAD_LOCATION}:/usr/src/app/upload
  #     - /etc/localtime:/etc/localtime:ro
  #   env_file:
  #     - .env
  #   depends_on:
  #     - redis
  #     - database
  #   restart: always

  immich-machine-learning:
    container_name: immich_machine_learning
    # For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag.
    # Example tag: ${IMMICH_VERSION:-release}-cuda
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration
      file: hwaccel.ml.yml
      service: openvino # set to one of [armnn, cuda, openvino, openvino-wsl] for accelerated inference - use the `-wsl` version for WSL2 where applicable
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: always

  redis:
    container_name: immich_redis
    image: docker.io/redis:6.2-alpine@sha256:d6c2911ac51b289db208767581a5d154544f2b2fe4914ea5056443f62dc6e900
    healthcheck:
      test: redis-cli ping || exit 1
    restart: always

  # database:
  #   container_name: immich_postgres
  #   image: registry.hub.docker.com/tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0
  #   environment:
  #     POSTGRES_PASSWORD: ${DB_PASSWORD}
  #     POSTGRES_USER: ${DB_USERNAME}
  #     POSTGRES_DB: ${DB_DATABASE_NAME}
  #   volumes:
  #     - pgdata:/var/lib/postgresql/data
  #   restart: always

  database:
    container_name: immich_postgres
    image: docker.io/tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      POSTGRES_INITDB_ARGS: '--data-checksums'
    volumes:
      - pgdata:/var/lib/postgresql/data
    healthcheck:
      test: pg_isready --dbname='${DB_DATABASE_NAME}' || exit 1; Chksum="$$(psql --dbname='${DB_DATABASE_NAME}' --username='${DB_USERNAME}' --tuples-only --no-align --command='SELECT COALESCE(SUM(checksum_failures), 0) FROM pg_stat_database')"; echo "checksum failure count is $$Chksum"; [ "$$Chksum" = '0' ] || exit 1
      interval: 5m
      start_interval: 30s
      start_period: 5m
    command: ["postgres", "-c" ,"shared_preload_libraries=vectors.so", "-c", 'search_path="$$user", public, vectors', "-c", "logging_collector=on", "-c", "max_wal_size=2GB", "-c", "shared_buffers=512MB", "-c", "wal_compression=on"]
    restart: always

volumes:
  pgdata:
  model-cache:

networks:
  default:
  lan:
    external: true
  tf0:
    external: true

Your .env content

# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

# The location where your uploaded files are stored
UPLOAD_LOCATION=/mnt/storage/photos/immich
# The location where your database files are stored
DB_DATA_LOCATION=./postgres

# To set a timezone, uncomment the next line and change Etc/UTC to a TZ identifier from this list: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List
TZ=America/Los_Angeles

# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=release

# Connection secret for postgres. You should change it to a random password
DB_PASSWORD=mypass

# The values below this line do not need to be changed
###################################################################################
DB_HOSTNAME=immich_postgres
DB_USERNAME=postgres
DB_DATABASE_NAME=immich

REDIS_HOSTNAME=immich_redis

Reproduction steps

1. Set two custom headers from Cloudflare service token.
2. Attempt to login via mobile app from external Internet.

Relevant log output

No response

Additional information

No response

bo0tzz commented 1 week ago

@matejkramny any ideas?

Sammy1Am commented 1 week ago

Quick update: I removed the headers from the settings and I get the same behavior, so it might not be directly header related (but still weird that I get a version mismatch and a 200 log entry).

I changed the URL of the server to one that doesn't exist and I get "Server is not reachable".

So my best theory so far is that when the app "pings" the server it's getting the Cloudflare login page, but thinks that's okay. And then for some reason doesn't like the "version"?

So it's possible it's a configuration issue on my tunnel, I'll have to test the service token in a browser later today and see what happens.

matejkramny commented 1 week ago

The 200 is the cloudflare login page. Any chance you are able to test connecting to immicha using the cloudflare headers with curl first?

Check the PR here I wrote some documentation around configuring the tokens: https://github.com/immich-app/immich/pull/10755 https://pr-10755.preview.immich.app/ Under guides -> cloudflare access

Sammy1Am commented 1 week ago

Tested with Firefox and it was just forwarding me to the login page even with the headers. But... then re-read the instructions and noticed I'd chosen "Allow" for the policy (you know, to allow connections) instead of "Service Auth". 😐

Anyway, working fine now. Thank you again for work on this!

matejkramny commented 1 week ago

Hi, yes that's a common pitfall - happened to me too. I'll make it clearer in the document

immich-app / immich