search

immich-app / immich

High performance self-hosted photo and video management solution.
https://immich.app
GNU Affero General Public License v3.0
49.64k stars 2.62k forks source link

[BUG] Photos with non-standard extensions can't be uploaded and preventing other photos from uploading on iOS #6173

Open mio-19 opened 9 months ago

mio-19 commented 9 months ago

The bug

Some app created jpeg files with problematic filenames like 71.pic in an iPad's Photo Library.

immich stopped uploading with the filename being **.pic, the progress bar at 100% and failure count at 1. I have to delete that photo or choose different albums to keep uploading, which makes uploading barely usable on this device.

error log on server

[Nest] 7  - 12/31/2023, 3:21:40 PM   ERROR [AssetService] Unsupported file type 71.pic

photo: https://github.com/immich-app/immich/files/13801455/71.pic.jpg.zip

The OS that Immich Server is running on

Archlinux

Version of Immich Server

1.91.4

Version of Immich Mobile App

1.91.4 on iPadOS 17

Platform with the issue

Your docker-compose.yml content

version: "3.8"

#
# WARNING: Make sure to use the docker-compose.yml of the current release:
#
# https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
#
# The compose file on main may not be compatible with the latest release.
#

name: immich

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    command: [ "start.sh", "immich" ]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    ports:
      - 2283:3001
    depends_on:
      - redis
      - database
    restart: always

  immich-microservices:
    container_name: immich_microservices
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    # extends:
    #   file: hwaccel.yml
    #   service: hwaccel
    command: [ "start.sh", "microservices" ]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    depends_on:
      - redis
      - database
    restart: always

  immich-machine-learning:
    container_name: immich_machine_learning
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: always

  redis:
    container_name: immich_redis
    image: redis:6.2-alpine@sha256:b6124ab2e45cc332e16398022a411d7e37181f21ff7874835e0180f56a09e82a
    restart: always

  database:
    container_name: immich_postgres
    image: tensorchord/pgvecto-rs:pg14-v0.1.11@sha256:0335a1a22f8c5dd1b697f14f079934f5152eaaa216c09b61e293be285491f8ee
    env_file:
      - .env
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
    volumes:
      - pgdata:/var/lib/postgresql/data
    restart: always

volumes:
  pgdata:
  model-cache:

Your .env content

# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

# The location where your uploaded files are stored
UPLOAD_LOCATION=/run/desktop/mnt/host/y/data1/immich-app/library

# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=v1.91.4

# Connection secret for postgres. You should change it to a random password
DB_PASSWORD=***********************

# The values below this line do not need to be changed
###################################################################################
DB_HOSTNAME=immich_postgres
DB_USERNAME=postgres
DB_DATABASE_NAME=immich

REDIS_HOSTNAME=immich_redis

Reproduction steps

See Above

Additional information

original one: https://github.com/immich-app/immich/issues/6028

alextran1502 commented 9 months ago

Do you know which app can create those file names? I want to reproduce this to solve the issue

mio-19 commented 9 months ago

Do you know which app can create those file names? I want to reproduce this to solve the issue

Wechat

mio-19 commented 9 months ago

I used iMessage to transfer another one of them to an iPhone and the extension became pict for some reason. Their extension is exactly pic on the iCloud Data Export.

$ find -name '*.pic*'
./iCloud Photos Part 12 of 29/Photos/306.pic
./iCloud Photos Part 12 of 29/Photos/7285.pic
./iCloud Photos Part 11 of 29/Photos/7440.pic
./iCloud Photos Part 11 of 29/Photos/7772.pic
./iCloud Photos Part 11 of 29/Photos/69.pic
./iCloud Photos Part 11 of 29/Photos/711.pic
./iCloud Photos Part 11 of 29/Photos/71.pic
./iCloud Photos Part 11 of 29/Photos/160.pic
./iCloud Photos Part 11 of 29/Photos/305.pic
gorootde commented 9 months ago

Same issue with files ending with .rw2 (Panasonic Camera RAW Files). They display properly (even in the error list) but do not upload to immich server.

mio-19 commented 9 months ago

Do you know which app can create those file names? I want to reproduce this to solve the issue

The app only created such files on one device. I don't have problems on another device with this app. Maybe gorootde's case is more reproducible

mertalev commented 9 months ago

The only reference I can see to .pic files is the PICT format that's been deprecated since like 2005. Is this actually what's being generated here, or is it just a JPEG file with a weird extension?

mio-19 commented 9 months ago

The only reference I can see to .pic files is the PICT format that's been deprecated since like 2005. Is this actually what's being generated here, or is it just a JPEG file with a weird extension?

It is just a JPEG file

mio-19 commented 8 months ago

I encounter this issue again. Is it an individual issue that the bakup procedure is stopped and not uploading other photos?

mertalev commented 6 months ago

No, the app does stop backing up if it fails for an image. That should be changed to continue backing up the other assets.

asac51 commented 6 months ago

I encounter the same issue, with *.pic file from Wechat on my iPhone.

benjR commented 6 months ago

Originally opened a discussion here and found this issue while searching for a way to fix it, as I encounter the same issue but by uploading by bulk using the CLI tool.

On my side, just "message":"Unsupported file type .jpg", which doesn't help either to find which file is blocking the import (no name, no path), in the same fashion the error should not prevent the import from continuing.

serghov commented 5 months ago

I encountered the same issue on android. A single .ts file blocks the whole backup process.

mattnovelli commented 5 months ago

An Adobe Illustrator .ai file is blocking my backup.

eng1n88r commented 2 months ago

I had the same issue with the photos from DJI Fly app. Checked it on the phone and it is regular JPEG: image

And in the server logs:

[Nest] 17  - 08/14/2024, 11:59:35 PM   ERROR [Api:AssetMediaService~xpugdfhp] Unsupported file type dji_fly_20230420_162830_184_1682022581430_photo_optimized
saqwerty commented 1 month ago

It block the whole backup process.

[Nest] 17 - 09/16/2024, 6:43:18 PM ERROR [Api:AssetMediaService~0f6bwcdz] Unsupported file type IMG_20240409011411932

alextran1502 commented 1 month ago

Can you guys help provide a way to create those unsupported file extension from the phone so I can help fix this issue?

On Mon, Sep 16, 2024 at 1:53 PM saqwerty @.***> wrote:

It block the whole backup process.

[Nest] 17 - 09/16/2024, 6:43:18 PM ERROR [Api:AssetMediaService~0f6bwcdz] Unsupported file type IMG_20240409011411932

— Reply to this email directly, view it on GitHub https://github.com/immich-app/immich/issues/6173#issuecomment-2353673189, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGONL7QVWZZRFVTAYIY25ELZW4SJDAVCNFSM6AAAAABBNMGN72VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNJTGY3TGMJYHE . You are receiving this because you commented.Message ID: @.***>

saqwerty commented 1 month ago

Can you guys help provide a way to create those unsupported file extension from the phone so I can help fix this issue? … On Mon, Sep 16, 2024 at 1:53 PM saqwerty @.> wrote: It block the whole backup process. [Nest] 17 - 09/16/2024, 6:43:18 PM ERROR [Api:AssetMediaService~0f6bwcdz] Unsupported file type IMG_20240409011411932 — Reply to this email directly, view it on GitHub <#6173 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGONL7QVWZZRFVTAYIY25ELZW4SJDAVCNFSM6AAAAABBNMGN72VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNJTGY3TGMJYHE . You are receiving this because you commented.Message ID: @.>

My kid use a wink app to create these images, she think they were exported from a video by the app, but she can't reproduce it anymore. These images just don't have extension name.
Maybe you can forcely change the file extention name in the code to simuate it. The root problem is that if there is a error the whole uploading process should not be blocked, and just skip the image and keep the process going, and it will be even better if these unuploaded images can be put a temp “error" album so it can be easily checked manully.

fullburnen commented 3 weeks ago

To get around the WeChat .pic files blocking uploads, I have moved them to a WeChat album in iOS and excluded that album from the immich backup list. Not the best solution, but at least it won't block uploading my camera roll.