immich-app / immich

High performance self-hosted photo and video management solution.
https://immich.app
GNU Affero General Public License v3.0
51.95k stars 2.75k forks source link

Upload via Domain -> Cloudflare is crashing #10001

Closed kempfdaniel closed 5 months ago

kempfdaniel commented 5 months ago

The bug

Hey, First of all I wanted to say that this is a great project and I hope it gets more attention, even if it is still a bit rough here and there.

I wanted to test synchronising all my photos from my mobile phone via the app, which didn't work for all photos and videos, such as the file mentioned above. So I just wanted to do the upload manually and downloaded the video from the icloud and wanted to upload it via the web interface (via the domain) which didn't work and then tried it via the cli and then got this error that the file was too large via Cloudflare. So I just found out that I can upload files via IP via cli and via webui but not via domain which is routed via cloudflare.

Here are the error logs:

PS C:\Users\dk\Downloads> ls .\IMG_0502.MP4

    Directory: C:\Users\dk\Downloads

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
-a---          05.06.2024    20:49      164650738 IMG_0502.MP4

PS C:\Users\dk\Downloads> immich -u https://photos.<privat>.de -k <RandomToken> upload .\IMG_0502.MP4
Discovered API at https://photos.<privat>.de/api
Crawling for assets...
Checking files | ████████████████████████████████████████ | 100% | ETA: 0s | 1/1 assets
Found 1 new files and 0 duplicates
Uploading assets | ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ | 0% | ETA: 0s | 0/164,7 MB
file:///C:/Users/dk/AppData/Roaming/nvm/v21.5.0/node_modules/@immich/cli/dist/index.js:17931
    throw new Error(await response.text());
          ^

Error: <html>
<head><title>413 Request Entity Too Large</title></head>
<body>
<center><h1>413 Request Entity Too Large</h1></center>
<hr><center>cloudflare</center>
<script>(function(){if (!document.body) return;var js = "window['__CF$cv$params']={r:'88f268ec8f21b78b',t:'MTcxNzYxNDA3MS43NzMwMDA='};_cpo=document.createElement('script');_cpo.nonce='',_cpo.src='/cdn-cgi/challenge-platform/scripts/jsd/main.js',document.getElementsByTagName('head')[0].appendChild(_cpo);";var _0xh = document.createElement('iframe');_0xh.height = 1;_0xh.width = 1;_0xh.style.position = 'absolute';_0xh.style.top = 0;_0xh.style.left = 0;_0xh.style.border = 'none';_0xh.style.visibility = 'hidden';document.body.appendChild(_0xh);function handler() {var _0xi = _0xh.contentDocument || _0xh.contentWindow.document;if (_0xi) {var _0xj = _0xi.createElement('script');_0xj.innerHTML = js;_0xi.getElementsByTagName('head')[0].appendChild(_0xj);}}if (document.readyState !== 'loading') {handler();} else if (window.addEventListener) {document.addEventListener('DOMContentLoaded', handler);} else {var prev = document.onreadystatechange || function () {};document.onreadystatechange = function (e) {prev(e);if (document.readyState !== 'loading') {document.onreadystatechange = prev;handler();}};}})();</script></body>
</html>

    at uploadFile (file:///C:/Users/dk/AppData/Roaming/nvm/v21.5.0/node_modules/@immich/cli/dist/index.js:17931:11)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async file:///C:/Users/dk/AppData/Roaming/nvm/v21.5.0/node_modules/@immich/cli/dist/index.js:17883:28
    at async Promise.all (index 0)
    at async uploadFiles (file:///C:/Users/dk/AppData/Roaming/nvm/v21.5.0/node_modules/@immich/cli/dist/index.js:17880:7)
    at async upload (file:///C:/Users/dk/AppData/Roaming/nvm/v21.5.0/node_modules/@immich/cli/dist/index.js:17811:21)

Node.js v21.5.0
PS C:\Users\dk\Downloads> immich -u http://192.168.178.125:2283 -k <RandomToken> upload .\IMG_0502.MP4
Discovered API at http://192.168.178.125:2283/api
Crawling for assets...
Checking files | ████████████████████████████████████████ | 100% | ETA: 0s | 1/1 assets
Found 1 new files and 0 duplicates
Uploading assets | ████████████████████████████████████████ | 100% | ETA: 0s | 164,7 MB/164,7 MB
Successfully uploaded 1 asset (164,7 MB)
PS C:\Users\dk\Downloads>

The OS that Immich Server is running on

Linux rpi 6.6.31+rpt-rpi-v8 #1 SMP PREEMPT Debian 1:6.6.31-1+rpt1 (2024-05-29) aarch64 GNU/Linux

Version of Immich Server

1.105.1

Version of Immich Mobile App

1.105.0 build.157

Platform with the issue

Your docker-compose.yml content

#
# WARNING: Make sure to use the docker-compose.yml of the current release:
#
# https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
#
# The compose file on main may not be compatible with the latest release.
#

name: immich

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    command: ['start.sh', 'immich']
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    ports:
      - 2283:3001
    depends_on:
      - redis
      - database
    restart: always

  immich-microservices:
    container_name: immich_microservices
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    # extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/hardware-transcoding
    #   file: hwaccel.transcoding.yml
    #   service: cpu # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding
    command: ['start.sh', 'microservices']
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    depends_on:
      - redis
      - database
    restart: always

  immich-machine-learning:
    container_name: immich_machine_learning
    # For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag.
    # Example tag: ${IMMICH_VERSION:-release}-cuda
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    # extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration
    #   file: hwaccel.ml.yml
    #   service: cpu # set to one of [armnn, cuda, openvino, openvino-wsl] for accelerated inference - use the `-wsl` version for WSL2 where applicable
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: always

  redis:
    container_name: immich_redis
    image: registry.hub.docker.com/library/redis:6.2-alpine@sha256:84882e87b54734154586e5f8abd4dce69fe7311315e2fc6d67c29614c8de2672
    restart: always

  database:
    container_name: immich_postgres
    image: registry.hub.docker.com/tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      POSTGRES_INITDB_ARGS: '--data-checksums'
    volumes:
      - ${DB_DATA_LOCATION}:/var/lib/postgresql/data
    ports:
      - 5432:5432
    restart: always
    command: ["postgres", "-c" ,"shared_preload_libraries=vectors.so", "-c", 'search_path="$$user", public, vectors', "-c", "logging_collector=on", "-c", "max_wal_size=2GB", "-c", "shared_buffers=512MB", "-c", "wal_compression=on"]

volumes:
  model-cache:

Your .env content

no special changes

Reproduction steps

1. Provide a file <= 100MB (Cloudflare Free is 100MB included business 200MB)
2. Upload Map to immich server over cli or webui using the domain linked to cloudflare
3. See Error Message

Relevant log output

PS C:\Users\dk\Downloads> ls .\IMG_0502.MP4

    Directory: C:\Users\dk\Downloads

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
-a---          05.06.2024    20:49      164650738 IMG_0502.MP4

PS C:\Users\dk\Downloads> immich -u https://photos.<privat>.de -k <RandomToken> upload .\IMG_0502.MP4
Discovered API at https://photos.<privat>.de/api
Crawling for assets...
Checking files | ████████████████████████████████████████ | 100% | ETA: 0s | 1/1 assets
Found 1 new files and 0 duplicates
Uploading assets | ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ | 0% | ETA: 0s | 0/164,7 MB
file:///C:/Users/dk/AppData/Roaming/nvm/v21.5.0/node_modules/@immich/cli/dist/index.js:17931
    throw new Error(await response.text());
          ^

Error: <html>
<head><title>413 Request Entity Too Large</title></head>
<body>
<center><h1>413 Request Entity Too Large</h1></center>
<hr><center>cloudflare</center>
<script>(function(){if (!document.body) return;var js = "window['__CF$cv$params']={r:'88f268ec8f21b78b',t:'MTcxNzYxNDA3MS43NzMwMDA='};_cpo=document.createElement('script');_cpo.nonce='',_cpo.src='/cdn-cgi/challenge-platform/scripts/jsd/main.js',document.getElementsByTagName('head')[0].appendChild(_cpo);";var _0xh = document.createElement('iframe');_0xh.height = 1;_0xh.width = 1;_0xh.style.position = 'absolute';_0xh.style.top = 0;_0xh.style.left = 0;_0xh.style.border = 'none';_0xh.style.visibility = 'hidden';document.body.appendChild(_0xh);function handler() {var _0xi = _0xh.contentDocument || _0xh.contentWindow.document;if (_0xi) {var _0xj = _0xi.createElement('script');_0xj.innerHTML = js;_0xi.getElementsByTagName('head')[0].appendChild(_0xj);}}if (document.readyState !== 'loading') {handler();} else if (window.addEventListener) {document.addEventListener('DOMContentLoaded', handler);} else {var prev = document.onreadystatechange || function () {};document.onreadystatechange = function (e) {prev(e);if (document.readyState !== 'loading') {document.onreadystatechange = prev;handler();}};}})();</script></body>
</html>

    at uploadFile (file:///C:/Users/dk/AppData/Roaming/nvm/v21.5.0/node_modules/@immich/cli/dist/index.js:17931:11)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async file:///C:/Users/dk/AppData/Roaming/nvm/v21.5.0/node_modules/@immich/cli/dist/index.js:17883:28
    at async Promise.all (index 0)
    at async uploadFiles (file:///C:/Users/dk/AppData/Roaming/nvm/v21.5.0/node_modules/@immich/cli/dist/index.js:17880:7)
    at async upload (file:///C:/Users/dk/AppData/Roaming/nvm/v21.5.0/node_modules/@immich/cli/dist/index.js:17811:21)

Node.js v21.5.0
PS C:\Users\dk\Downloads> immich -u http://192.168.178.125:2283 -k <RandomToken> upload .\IMG_0502.MP4
Discovered API at http://192.168.178.125:2283/api
Crawling for assets...
Checking files | ████████████████████████████████████████ | 100% | ETA: 0s | 1/1 assets
Found 1 new files and 0 duplicates
Uploading assets | ████████████████████████████████████████ | 100% | ETA: 0s | 164,7 MB/164,7 MB
Successfully uploaded 1 asset (164,7 MB)
PS C:\Users\dk\Downloads>

Additional information

No response

bo0tzz commented 5 months ago

This is a cloudflare limit