search

immich-app / immich

High performance self-hosted photo and video management solution.
https://immich.app
GNU Affero General Public License v3.0
43.91k stars 2.14k forks source link

File download fails and hangs after exactly 512 MB for large file (Android only / no issue on web client) #6583

Open tnoms opened 7 months ago

tnoms commented 7 months ago

The bug

While trying to download a 980 MB video file using the Immich app on Android, the download consistently fails and hangs. In two consecutive tests, a partial video file exactly 512 MB in size was written to disk. In the third test, 100k error logs were generated about running out of memory. This issue does not appear in the web client.

Error logs show: "PlatformDispacter - Catch all error: Out of Memory"

Assumptions:

While brainstorming with a friend, we discovered the following Stack Overflow post noting that manufacturers can enforce heap limits on apps to prevent excessive memory usage. This could be related.

The OS that Immich Server is running on

Debian GNU/Linux 12 (bookworm)

Version of Immich Server

v1.92.1

Version of Immich Mobile App

v1.92.1

Platform with the issue

Your docker-compose.yml content

version: "3.8"

...

# Immich block 1 start #
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    command: [ "start.sh", "immich" ]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    ports:
      - 2283:3001
    depends_on:
      - redis
      - database
    restart: always

  immich-microservices:
    container_name: immich_microservices
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    # extends:
    #   file: hwaccel.yml
    #   service: hwaccel
    command: [ "start.sh", "microservices" ]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    depends_on:
      - redis
      - database
    restart: always

  immich-machine-learning:
    container_name: immich_machine_learning
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    volumes:
      - /docker/Immich/model-cache:/cache
    env_file:
      - .env
    restart: always

  redis:
    container_name: immich_redis
    image: redis:6.2-alpine@sha256:c5a607fb6e1bb15d32bbcf14db22787d19e428d59e31a5da67511b49bb0f1ccc
    restart: always

  database:
    container_name: immich_postgres
    image: tensorchord/pgvecto-rs:pg14-v0.1.11@sha256:0335a1a22f8c5dd1b697f14f079934f5152eaaa216c09b61e293be285491f8ee
    env_file:
      - .env
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
    volumes:
      - /docker/Immich/pgdata:/var/lib/postgresql/data
    restart: always
# Immich block 1 end #

  ...

# Immich block 2 start
volumes:
  pgdata:
  model-cache:
# Immich block 2 end

Your .env content

# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

# The location where your uploaded files are stored
UPLOAD_LOCATION=/media/Storage/.Immich

# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=release

# Connection secret for postgres. You should change it to a random password
DB_PASSWORD=postgres

# The values below this line do not need to be changed
###################################################################################
DB_HOSTNAME=immich_postgres
DB_USERNAME=postgres
DB_DATABASE_NAME=immich

REDIS_HOSTNAME=immich_redis

Reproduction steps

Initial tests performed:

  1. Android App Download - client hung, waited for a long time, killed the app, saw exactly 512 MB of the file on my phone
  2. Android App Download - client hung, waited for a long time, killed the app, saw exactly 512 MB of the file on my phone

Next test performed:

  1. Web Client Download - downloaded same video fully and quickly with no issues

Final test performed:

  1. Android App Download - client hung, waited for a long time, checked memory settings and noted elevated but not maxed out, Immich error log had generated ~100,000 out of memory logs, killed the app, file was never written to disk

Additional information

App was downloaded from F-Droid, not Google Play

kamzil commented 6 months ago

There's a similar issue in the web client in mobile browser: #7542

Like you said, the problem is likely because it loads the full file in RAM before saving to disk.

gizatulov commented 1 month ago

Facing exactly the same issue. MobileApp version: 1.107.1 Server version: 1.106.4 `#0 _ByteCallbackSink.add (dart:convert/byte_conversion.dart)

1 _RootZone.runUnaryGuarded (dart:async/zone.dart)

2 _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart)

3 _BufferingStreamSubscription._add (dart:async/stream_impl.dart)

4 _ForwardingStreamSubscription._add (dart:async/stream_pipe.dart)

5 _ForwardingStreamSubscription._handleData (dart:async/stream_pipe.dart)

6 _RootZone.runUnaryGuarded (dart:async/zone.dart)

7 _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart)

8 _BufferingStreamSubscription._add (dart:async/stream_impl.dart)

9 _ForwardingStreamSubscription._add (dart:async/stream_pipe.dart)

10 _ForwardingStreamSubscription._handleData (dart:async/stream_pipe.dart)

11 _RootZone.runUnaryGuarded (dart:async/zone.dart)

12 _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart)

13 _BufferingStreamSubscription._add (dart:async/stream_impl.dart)

14 _StreamController._add (dart:async/stream_controller.dart)

15 _HttpParser._doParse (dart:_http/http_parser.dart)

16 _HttpParser._parse (dart:_http/http_parser.dart)

17 _HttpParser._onData (dart:_http/http_parser.dart)

18 _RootZone.runUnaryGuarded (dart:async/zone.dart)

19 _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart)

20 _BufferingStreamSubscription._add (dart:async/stream_impl.dart)

21 _StreamController._add (dart:async/stream_controller.dart)

22 _StreamController.add (dart:async/stream_controller.dart)

23 _Socket._onData (dart:io-patch/socket_patch.dart)

24 _RootZone.runUnaryGuarded (dart:async/zone.dart)

25 _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart)

26 _BufferingStreamSubscription._add (dart:async/stream_impl.dart)

27 _StreamController._add (dart:async/stream_controller.dart)

28 _StreamController.add (dart:async/stream_controller.dart)

29 _RawSecureSocket._sendReadEvent (dart:io/secure_socket.dart)

30 Timer._createTimer. (dart:async-patch/timer_patch.dart)

31 _Timer._runTimers (dart:isolate-patch/timer_impl.dart)

32 _Timer._handleMessage (dart:isolate-patch/timer_impl.dart)

33 _RawReceivePort._handleMessage (dart:isolate-patch/isolate_patch.dart)`

sffetlio commented 1 month ago

Same here. Downloading 310 mb video file in the Android app throws out of memory exception and creates broken 256 mb file. No issues downloading in Brave app ver: 1.108.0 server: 1.108.0