search

Salvoxia / immich-folder-album-creator

Automatically create and populate albums in Immich from a folder structure in external libraries
https://hub.docker.com/r/salvoxia/immich-folder-album-creator
222 stars 13 forks source link

Main album unexpected name; Main album cannot be deleted #13

Closed mixpc closed 5 months ago

mixpc commented 5 months ago

@Salvoxia thank you for such a useful tool! Much needed!

I have a scenario where the main album with an unexpected album name and which holds all photos cannot be deleted. It may just be my wrongdoing, though.

In a setup where immich is running as a docker and an external library is set as well and mapped as...

home/user/USBdrive/Photos:/usr/src/app/external and

home/user/USBdrive/Photos
home/user/USBdrive/Photos/Family Photos
home/user/USBdrive/Photos/Family Photos/Trips
home/user/USBdrive/Photos/Family Photos/Trips/Trip 001
home/user/USBdrive/Photos/Family Photos/Trips/Trip 001
...
home/user/USBdrive/Photos/Family Photos/Trips/Trip 099

When running immich-folder-album-creator docker (Raspbian (Debian 12 bookworm)

sudo docker run --rm -e API_URL="https://website.com/api/" -e API_KEY="123" -e ROOT_PATH="/usr/src/app/external" -e ALBUM_LEVELS=3 -e LOG_LEVEL=INFO salvoxia/immich-folder-album-creator:latest /script/immich_auto_album.sh the following albums are created

usb src
Family Photos Trips Trip 001
...
Family Photos Trips Trip 099

Albums named "Family Photos Trips Trip xxx" can be deleted (one by one) but album "usb src" cannot. A message reports it cannot be deleted, the log shows:

[Nest] 6  - 05/19/2024, 11:39:37 AM    WARN [ImmichServer] [ExpressAdapter] Content-Type doesn't match Reply body, you might need a custom ExceptionFilter for non-JSON responses
[Nest] 6  - 05/19/2024, 11:40:57 AM   ERROR [ImmichServer] [QueryFailedError: bind message has 18802 parameter formats but 0 parameters
    at PostgresQueryRunner.query (/usr/src/app/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:219:19)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async DeleteQueryBuilder.execute (/usr/src/app/node_modules/typeorm/query-builder/DeleteQueryBuilder.js:52:33)
    at async SubjectExecutor.executeRemoveOperations (/usr/src/app/node_modules/typeorm/persistence/SubjectExecutor.js:461:17)
    at async SubjectExecutor.execute (/usr/src/app/node_modules/typeorm/persistence/SubjectExecutor.js:104:9)
    at async EntityPersistExecutor.execute (/usr/src/app/node_modules/typeorm/persistence/EntityPersistExecutor.js:140:21)
    at async AlbumRepository.delete (/usr/src/app/dist/repositories/album.repository.js:230:9)
    at async AlbumService.delete (/usr/src/app/dist/services/album.service.js:153:9)] Failed to delete album
[Nest] 6  - 05/19/2024, 11:40:57 AM   ERROR [ImmichServer] [QueryFailedError: bind message has 18802 parameter formats but 0 parameters
    at PostgresQueryRunner.query (/usr/src/app/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:219:19)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async DeleteQueryBuilder.execute (/usr/src/app/node_modules/typeorm/query-builder/DeleteQueryBuilder.js:52:33)
    at async SubjectExecutor.executeRemoveOperations (/usr/src/app/node_modules/typeorm/persistence/SubjectExecutor.js:461:17)
    at async SubjectExecutor.execute (/usr/src/app/node_modules/typeorm/persistence/SubjectExecutor.js:104:9)
    at async EntityPersistExecutor.execute (/usr/src/app/node_modules/typeorm/persistence/EntityPersistExecutor.js:140:21)
    at async AlbumRepository.delete (/usr/src/app/dist/repositories/album.repository.js:230:9)
    at async AlbumService.delete (/usr/src/app/dist/services/album.service.js:153:9)] QueryFailedError: bind message has 18802 parameter formats but 0 parameters

The assumption here was setting -e ROOT_PATH="/usr/src/app/external" as -e ROOT_PATH="/home/user/USBdrive/Photos" would not find the photos.

If I manually remove all albums and run

sudo docker run --rm -e API_URL="https://website.com/api/" -e API_KEY="123" -e ROOT_PATH="/usr/src/app/external/Family Photos" -e ALBUM_LEVELS=3 -e LOG_LEVEL=INFO salvoxia/immich-folder-album-creator:latest /script/immich_auto_album.sh It will still show album "usr src":

time=2024-05-19T10:40:16.713+00:00 level=INFO msg=Requesting all assets
time=2024-05-19T10:40:29.913+00:00 level=INFO msg=42169 photos found
time=2024-05-19T10:40:29.913+00:00 level=INFO msg=Sorting assets to corresponding albums using folder name
time=2024-05-19T10:40:30.098+00:00 level=INFO msg=1 albums identified
time=2024-05-19T10:40:30.098+00:00 level=INFO msg=Album list: [' usr src']
time=2024-05-19T10:40:30.098+00:00 level=INFO msg=Listing existing albums on immich
time=2024-05-19T10:40:30.393+00:00 level=INFO msg=1 existing albums identified
time=2024-05-19T10:40:30.393+00:00 level=INFO msg=Creating albums if needed
time=2024-05-19T10:40:30.393+00:00 level=INFO msg=0 albums created
time=2024-05-19T10:40:30.394+00:00 level=INFO msg=Adding assets to albums
time=2024-05-19T10:40:32.476+00:00 level=INFO msg=Done!

Is there a way to reset and begin from scratch? So that it correctly identifies the folder structure? It may have to do with setting the root path differently. Or a way to have the main album with all photos be properly named as "Family Photos" as per ROOT_PATH setting?

Thank you for any feedback in advance.

Salvoxia commented 5 months ago

Hi,

something is definitely off. First for the error when trying to delete the "usr src" album: I assume that all of your images have been added to that single album? If so, it looks you're affected by this Immich bug when trying to delete albums with a huge number of images. A workaround would be to remove images in chunks from the album until the number of images falls below the critical threshold. This script faced a similar issue when trying to add too many images to an album at once, and it seems the Immich web UI has the same problem for deleting.

Now for the strange album itself:
I'm not yet sure what's going on there. I tried recreating your setup with that exact mount path, and the albums the script wants to create are exactly what I expect, no "usr src" in there. What's also very strange is the leading blank in the album name

time=2024-05-19T10:40:30.098+00:00 level=INFO msg=Album list: [' usr src']

Could you please run the script again with -e LOG_LEVEL=DEBUG and post the logs here (make sure to run tag 0.5.0 or re-pull latest tag)? It will be VERY long for 42k files, but hopefully can shed some light on the reason for this strange behavior.

Also, is /usr/src/app/external the only Import Path you have set for your external library?

Best Regards, Salvoxia

mixpc commented 5 months ago

Hi,

Thank you for your time to review. Instead, I've chosen to rebuild immich from scractch, created external library and scanned new library files (machine learning, video transcoding disabled for quicker import). So this feedback basically is a new attempt.

Regarding album deletion, I note the immich bug, thank you for the link and feedback. And yes, /usr/src/app/external is the only Import Path for my external library

Server status online
Server version v1.105.1

sudo docker exec -it immich_server /bin/bash
immich login https://website.com/api apikey
root@ababababab:/usr/src/app# immich -V
2.2.0

command:

sudo docker run --rm -e API_URL="https://website.com/api/" -e API_KEY="thisistheapikey" -e ROOT_PATH="/usr/src/app/external" -e ALBUM_LEVELS=3 -e LOG_LEVEL=DEBUG salvoxia/immich-folder-album-creator:latest /script/immich_auto_album.sh

For the album, I'm run the script again with the DEBUG parameter, it's big file, 11MB. But the good thing is this time not a single error, no strange album at all, the exact 150 albums have been created. I must check that each album has the right number of photos, but I'm sure that will be ok.

time=2024-05-20T13:09:22.532+00:00 level=DEBUG msg=root_path = ['/usr/src/app/external']
time=2024-05-20T13:09:22.532+00:00 level=DEBUG msg=root_url = https://website.com/api/
time=2024-05-20T13:09:22.532+00:00 level=DEBUG msg=api_key = thisistheapikey
time=2024-05-20T13:09:22.532+00:00 level=DEBUG msg=number_of_images_per_request = 2000
time=2024-05-20T13:09:22.532+00:00 level=DEBUG msg=number_of_assets_to_fetch_per_request = 5000
time=2024-05-20T13:09:22.532+00:00 level=DEBUG msg=unattended = True
time=2024-05-20T13:09:22.532+00:00 level=DEBUG msg=album_levels = 3
time=2024-05-20T13:09:22.532+00:00 level=DEBUG msg=album_level_separator =
time=2024-05-20T13:09:22.532+00:00 level=INFO msg=Requesting all assets
time=2024-05-20T13:09:22.534+00:00 level=DEBUG msg=Starting new HTTPS connection (1): website.com:443
time=2024-05-20T13:09:27.745+00:00 level=DEBUG msg=https://website.com:443 "GET /api/asset?take=5000 HTTP/1.1" 200 6702353
time=2024-05-20T13:09:27.880+00:00 level=DEBUG msg=Received 5000 assets with chunk 1
time=2024-05-20T13:09:28.086+00:00 level=DEBUG msg=Starting new HTTPS connection (1): website.com:443
time=2024-05-20T13:09:30.492+00:00 level=DEBUG msg=https://website.com:443 "GET /api/asset?take=5000&skip=5000 HTTP/1.1" 200 6638550
time=2024-05-20T13:09:30.729+00:00 level=DEBUG msg=Received 5000 assets with chunk
time=2024-05-20T13:09:30.917+00:00 level=DEBUG msg=Starting new HTTPS connection (1): website.com:443
time=2024-05-20T13:09:32.026+00:00 level=DEBUG msg=https://website.com:443 "GET /api/asset?take=5000&skip=10000 HTTP/1.1" 200 6753833
time=2024-05-20T13:09:32.159+00:00 level=DEBUG msg=Received 5000 assets with chunk
time=2024-05-20T13:09:32.360+00:00 level=DEBUG msg=Starting new HTTPS connection (1): website.com:443
time=2024-05-20T13:09:37.423+00:00 level=DEBUG msg=https://website.com:443 "GET /api/asset?take=5000&skip=15000 HTTP/1.1" 200 6755903
time=2024-05-20T13:09:37.591+00:00 level=DEBUG msg=Received 5000 assets with chunk
time=2024-05-20T13:09:37.785+00:00 level=DEBUG msg=Starting new HTTPS connection (1): website.com:443
time=2024-05-20T13:09:38.901+00:00 level=DEBUG msg=https://website.com:443 "GET /api/asset?take=5000&skip=20000 HTTP/1.1" 200 6822480
time=2024-05-20T13:09:39.035+00:00 level=DEBUG msg=Received 5000 assets with chunk
time=2024-05-20T13:09:39.252+00:00 level=DEBUG msg=Starting new HTTPS connection (1): website.com:443
time=2024-05-20T13:09:41.843+00:00 level=DEBUG msg=https://website.com:443 "GET /api/asset?take=5000&skip=25000 HTTP/1.1" 200 6778459
time=2024-05-20T13:09:41.977+00:00 level=DEBUG msg=Received 5000 assets with chunk
time=2024-05-20T13:09:42.174+00:00 level=DEBUG msg=Starting new HTTPS connection (1): website.com:443
time=2024-05-20T13:09:43.265+00:00 level=DEBUG msg=https://website.com:443 "GET /api/asset?take=5000&skip=30000 HTTP/1.1" 200 6750336
time=2024-05-20T13:09:43.402+00:00 level=DEBUG msg=Received 5000 assets with chunk
time=2024-05-20T13:09:43.595+00:00 level=DEBUG msg=Starting new HTTPS connection (1): website.com:443
time=2024-05-20T13:09:44.685+00:00 level=DEBUG msg=https://website.com:443 "GET /api/asset?take=5000&skip=35000 HTTP/1.1" 200 6753376
time=2024-05-20T13:09:44.820+00:00 level=DEBUG msg=Received 5000 assets with chunk
time=2024-05-20T13:09:45.010+00:00 level=DEBUG msg=Starting new HTTPS connection (1): website.com:443
time=2024-05-20T13:09:45.595+00:00 level=DEBUG msg=https://website.com:443 "GET /api/asset?take=5000&skip=40000 HTTP/1.1" 200 2925378
time=2024-05-20T13:09:45.653+00:00 level=DEBUG msg=Received 2169 assets with chunk
time=2024-05-20T13:09:45.764+00:00 level=INFO msg=42169 photos found
...
time=2024-05-20T13:09:48.090+00:00 level=INFO msg=150 albums identified
time=2024-05-20T13:09:48.091+00:00 level=INFO msg=Album list: ['aaa', '...', 'zzz']
time=2024-05-20T13:09:48.091+00:00 level=INFO msg=Listing existing albums on immich
time=2024-05-20T13:09:48.092+00:00 level=DEBUG msg=Starting new HTTPS connection (1): website.com:443
time=2024-05-20T13:09:48.149+00:00 level=DEBUG msg=https://website.com:443 "GET /api/album HTTP/1.1" 200 2
time=2024-05-20T13:09:48.151+00:00 level=INFO msg=0 existing albums identified
time=2024-05-20T13:09:48.151+00:00 level=INFO msg=Creating albums if needed
...
time=2024-05-20T13:09:58.820+00:00 level=INFO msg=150 albums created
time=2024-05-20T13:09:58.820+00:00 level=INFO msg=Adding assets to albums
...
time=2024-05-20T13:10:19.053+00:00 level=INFO msg=Done!

Why was that strange album created? Really, I can't tell. But with a new install, it works as expected and your script is wonderful :)

Salvoxia commented 5 months ago

Thanks for reporting back! I would have loved to get behind whatever caused the initial problem, just to make sure it's not a systemic issue, but glad to hear it's working as expected after the reset. Closing the issue.