search

immich-app / immich

High performance self-hosted photo and video management solution.
https://immich.app
GNU Affero General Public License v3.0
46.35k stars 2.3k forks source link

[BUG] Various errors from the logcat that may be the cause of why some of my items aren't uploading #671

Closed JVT038 closed 1 year ago

JVT038 commented 2 years ago

Describe the bug So I've noticed that a couple of items are constantly being uploaded, while they already should've been uploaded. Also, these items aren't appearing on the website nor in the app, so they probably haven't been uploaded, which is probably why they're uploaded again. Due to the uploading problems, I decided to plug my phone in my PC and check the logcat of immich, and I saw some interesting stuff that may help debugging this. I am no Android / Kotlin / Java programmer, so I'm not sure what these errors mean, but they are there, and probably cause the uploading issues.

Task List

Please complete the task list below. We need this information to help us reproduce the bug or point out problems in your setup. You are not providing enough info may delay our effort to help you.

To Reproduce Steps to reproduce the behavior:

  1. Start upload of a file
  2. Attach your phone with adb to your computer
  3. Execute adb shell "logcat --pid=$(pidof -s app.alextran.immich)"
  4. See errors

Expected behavior I expected my items to be uploaded properly and not seeing any weird errors.

System

Additional context So I have various errors, and I don't know whether they're related to the uploading problem or not, but here are the logs from logcat. PS. A lot of these errors are repeated quite a lot, I'm just showing all errors once. Also, some files are being uploaded, or at least, in the redis log it occasionally says there have been 100 changes in 300 seconds, so I'm assuming there is actually stuff being uploaded. Perhaps I'm wrong, I don't know.

09-10 21:40:32.808 17384 17414 I flutter : [_updateAlbumBackupTime] failed to find album in state
09-10 21:40:32.871 17384 17414 I flutter : [WEBSOCKET] Attempting to connect to ws
09-10 21:40:33.306 17384 17448 W Glide   : Failed to find GeneratedAppGlideModule. You should include an annotationProcessor compile dependency on com.github.bumptech.glide:compiler in your application and a @GlideModule annotated AppGlideModule implementation or LibraryGlideModules will be silently ignored
09-10 21:40:33.736 17384 17458 D skia    : --- Failed to create image decoder with message 'unimplemented'
09-10 21:40:35.265 17384 17458 D ExifInterface: No image meets the size requirements of a thumbnail image.
09-10 21:40:37.731 17384 17458 E Bitmap  : OOM allocating Bitmap with dimensions 512 x 3835904
09-10 21:40:38.227 17384 17466 D skia    : --- Failed to create image decoder with message 'unimplemented'
09-10 21:57:31.930 17384 17414 I flutter : ERROR backupAsset: SocketException: Broken pipe (OS Error: Broken pipe, errno = 32), address = immich.mydomain.com, port = 41696
09-10 21:58:01.932 17384 17414 I flutter : ERROR backupAsset: SocketException: Write failed (OS Error: Broken pipe, errno = 32), address = immich.mydomain.com, port = 44734
09-10 21:42:07.080 17384 17414 I flutter : <html>
09-10 21:42:07.080 17384 17414 I flutter : ^
09-10 21:42:07.080 17384 17414 I flutter :
09-10 21:42:25.469 17384 17414 I flutter : ERROR backupAsset: FormatException: Unexpected character (at character 1)
09-10 21:50:31.486 17384 17404 I WM-WorkerWrapper: Work [ id=66494ba4-b7ee-42f0-92c0-4da230152ef5, tags={ app.alextran.immich.BackupWorker } ] was cancelled
09-10 21:50:31.486 17384 17404 I WM-WorkerWrapper: java.util.concurrent.CancellationException: Task was cancelled.
09-10 21:50:31.486 17384 17404 I WM-WorkerWrapper:      at androidx.work.impl.utils.futures.a.c(Unknown Source:2)
09-10 21:50:31.486 17384 17404 I WM-WorkerWrapper:      at androidx.work.impl.utils.futures.a.i(Unknown Source:30)
09-10 21:50:31.486 17384 17404 I WM-WorkerWrapper:      at androidx.work.impl.utils.futures.a.get(Unknown Source:21)
09-10 21:50:31.486 17384 17404 I WM-WorkerWrapper:      at w.j$b.run(Unknown Source:4)
09-10 21:50:31.486 17384 17404 I WM-WorkerWrapper:      at e0.g$a.run(Unknown Source:2)
09-10 21:50:31.486 17384 17404 I WM-WorkerWrapper:      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
09-10 21:50:31.486 17384 17404 I WM-WorkerWrapper:      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
09-10 21:50:31.486 17384 17404 I WM-WorkerWrapper:      at java.lang.Thread.run(Thread.java:923)
09-10 21:50:31.667 17384 18212 E flutter : [ERROR:flutter/shell/common/shell.cc(93)] Dart Error: Dart_LookupLibrary: library 'package:immich_mobile/modules/backup/background_service/background.service.dart' not found.
09-10 21:50:31.667 17384 18212 E flutter : [ERROR:flutter/runtime/dart_isolate.cc(668)] Could not resolve main entrypoint function.
09-10 21:50:31.667 17384 18212 E flutter : [ERROR:flutter/runtime/dart_isolate.cc(167)] Could not run the run main Dart entrypoint.
09-10 21:50:31.669 17384 18212 E flutter : [ERROR:flutter/runtime/runtime_controller.cc(385)] Could not create root isolate.
09-10 21:50:31.669 17384 18212 E flutter : [ERROR:flutter/shell/common/shell.cc(604)] Could not launch engine with configuration

Also, I use nginx-proxy-manager as a reverse proxy, with client_max_body_size 0; in advanced configuration to allow large file uploads. I'm not sure if my .env and/or docker-compose.yml file will help with debugging, but here it is:

DB_HOSTNAME=/sockets
DB_USERNAME=postgres
DB_PASSWORD=postgres
DB_DATABASE_NAME=immich
DB_PORT=5433
REDIS_HOSTNAME=immich_redis
REDIS_PASSWORD="mypassword"
REDIS_SOCKET=/sockets/immich_redis.sock
UPLOAD_LOCATION=/path/to/upload/location
JWT_SECRET=mysecret
ENABLE_MAPBOX=true
MAPBOX_KEY=mypublickey
version: "3.8"

services:
  immich-server:
    image: altran1502/immich-server:release
    entrypoint: ["/bin/sh", "./start-server.sh"]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - sockets:/sockets
    env_file:
      - .env
    environment:
      - NODE_ENV=production
    depends_on:
      - redis
      - database
    restart: always

  immich-microservices:
    image: altran1502/immich-server:release
    entrypoint: ["/bin/sh", "./start-microservices.sh"]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - sockets:/sockets
    env_file:
      - .env
    environment:
      - NODE_ENV=production
    depends_on:
      - redis
      - database
    restart: always

  # I disabled this container because my CPU doesn't have AVX thingy
  # immich-machine-learning:
  #   image: altran1502/immich-machine-learning:release
  #   entrypoint: ["/bin/sh", "./entrypoint.sh"]
  #   volumes:
  #     - ${UPLOAD_LOCATION}:/usr/src/app/upload
  #   env_file:
  #     - .env
  #   environment:
  #     - NODE_ENV=production
  #   depends_on:
  #     - database
  #   restart: always

  immich-web:
    image: altran1502/immich-web:release
    entrypoint: ["/bin/sh", "./entrypoint.sh"]
    env_file:
      - .env
    restart: always

  redis:
    container_name: immich_redis
    image: redis:6.2
    restart: always
    command: redis-server /etc/redis.conf
    volumes:
      - sockets:/sockets
      - $DOCKERDIR/immich/redis.conf:/etc/redis.conf
      - /etc/TZ:/etc/timezone:ro
      - $DOCKERDIR/immich/redisdb:/db

  database:
    container_name: immich_postgres
    image: postgres:14-bullseye
    user: 1026:100
    command: postgres -c unix_socket_directories='/var/run/postgresql/,/sockets/'
    env_file:
      - .env
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      PG_DATA: /var/lib/postgresql/data
      PGPORT: ${DB_PORT}
    volumes:
      - $DOCKERDIR/immich/pgdata:/var/lib/postgresql/data
      - sockets:/sockets
    restart: always

  immich-proxy:
    container_name: immich_proxy
    image: altran1502/immich-proxy:release
    ports:
      - 2283:80
    logging:
      driver: none
    depends_on:
      - immich-server
    restart: always

# I attached this volume, because I use unix sockets for postgres & redis instead of TCP
volumes:
  sockets:
    external: true
alextran1502 commented 2 years ago

Can you update the app to 1.28.2 so that the background upload is fixed?

There are two reasons the file might not be uploaded.

It could be unsupported file type or duplicated file. We have the mechanism to dedup on the server side so it would reject the duplicate file. We are working kn a mechanism to not upload the duplicate file anymore on the client.

JVT038 commented 2 years ago

I would update to 1.28.2, if I could, because I don't see an apk in the releases nor on f droid

JVT038 commented 2 years ago

Can you add a downloadable apk in the Github releases?

alextran1502 commented 2 years ago

Done. Note that installing the app from the APK might change the device ID which lead to incorrect files to be recognized as uploaded or non-uploaded on the device.

JVT038 commented 2 years ago

I updated and I haven't seen this error anymore:

09-10 21:50:31.486 17384 17404 I WM-WorkerWrapper: Work [ id=66494ba4-b7ee-42f0-92c0-4da230152ef5, tags={ app.alextran.immich.BackupWorker } ] was cancelled
09-10 21:50:31.486 17384 17404 I WM-WorkerWrapper: java.util.concurrent.CancellationException: Task was cancelled.
09-10 21:50:31.486 17384 17404 I WM-WorkerWrapper:      at androidx.work.impl.utils.futures.a.c(Unknown Source:2)
09-10 21:50:31.486 17384 17404 I WM-WorkerWrapper:      at androidx.work.impl.utils.futures.a.i(Unknown Source:30)
09-10 21:50:31.486 17384 17404 I WM-WorkerWrapper:      at androidx.work.impl.utils.futures.a.get(Unknown Source:21)
09-10 21:50:31.486 17384 17404 I WM-WorkerWrapper:      at w.j$b.run(Unknown Source:4)
09-10 21:50:31.486 17384 17404 I WM-WorkerWrapper:      at e0.g$a.run(Unknown Source:2)
09-10 21:50:31.486 17384 17404 I WM-WorkerWrapper:      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
09-10 21:50:31.486 17384 17404 I WM-WorkerWrapper:      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
09-10 21:50:31.486 17384 17404 I WM-WorkerWrapper:      at java.lang.Thread.run(Thread.java:923)
09-10 21:50:31.667 17384 18212 E flutter : [ERROR:flutter/shell/common/shell.cc(93)] Dart Error: Dart_LookupLibrary: library 'package:immich_mobile/modules/backup/background_service/background.service.dart' not found.
09-10 21:50:31.667 17384 18212 E flutter : [ERROR:flutter/runtime/dart_isolate.cc(668)] Could not resolve main entrypoint function.
09-10 21:50:31.667 17384 18212 E flutter : [ERROR:flutter/runtime/dart_isolate.cc(167)] Could not run the run main Dart entrypoint.
09-10 21:50:31.669 17384 18212 E flutter : [ERROR:flutter/runtime/runtime_controller.cc(385)] Could not create root isolate.
09-10 21:50:31.669 17384 18212 E flutter : [ERROR:flutter/shell/common/shell.cc(604)] Could not launch engine with configuration

But the other errors still persist and it still doesn't upload.

alextran1502 commented 2 years ago

It doesn't upload in the background or foreground?

JVT038 commented 2 years ago

Both. It doesn't upload in the background, nor in the foreground. Every time it reaches 100%, one of these errors are shown:

09-10 21:40:35.265 17384 17458 D ExifInterface: No image meets the size requirements of a thumbnail image.
09-10 21:57:31.930 17384 17414 I flutter : ERROR backupAsset: SocketException: Broken pipe (OS Error: Broken pipe, errno = 32), address = immich.mydomain.com, port = 41696
09-10 21:58:01.932 17384 17414 I flutter : ERROR backupAsset: SocketException: Write failed (OS Error: Broken pipe, errno = 32), address = immich.mydomain.com, port = 44734
09-10 21:40:33.736 17384 17458 D skia    : --- Failed to create image decoder with message 'unimplemented'

The files themselves aren't corrupted or anything. They play fine on my phone and my PC. Nothing is shown in the server logs either, so it's just on the mobile side.

alextran1502 commented 2 years ago

What is the file type? is the file something you can share?

manuke42 commented 2 years ago

I had similar issues with a nginx reverse proxy. The logcat didn't show the return message of the server as it could not be Json decoded (html page, thus the FormatException). Maybe in the code there could be a try catch block for this error to get the actual server response into the mobile logs? This line: https://github.com/immich-app/immich/blob/4271e24e59f190099e2bbf44c946efd1675544a1/mobile/lib/modules/backup/services/backup.service.dart#L240

For me it was a problem with maximal upload by client, which you already excluded. Maybe it's some other nginx config.

alextran1502 commented 1 year ago

is still issue still relevant?