Closed SitramSoft closed 9 months ago
Similar boat, wanting to move from the AIO install to the new compose; but hitting this limit on resources; I also noticed that it doesn't give a progress bar like the old CLI tool did, not sure if that is a bug or by design, not a huge issue.
Same problem here. First the CLI is consuming huge ammount of memory depending on the image count. In my case ~17.5 GB for aorund 11k images. Second issue: After the copy process is started, it gets slower and slower. I Aborted the process after around 1GB uploaded. Then the speed was around one pictures every 5 second. The first pictures copies fast as usual.
I tried this with the CLI directly in the Immich Container and also on my windows machine where i installed the CLI through npm.
Definitely something weird going on. Tried running on smaller individual folders and it will crash my entire Proxmox server with CPU stalls but am not able to see anything in any logs. Resources don't seem to spike, that I notice.
@esmoyer how many ram do you have on the machine you are trying to run the cli on?
4 CPU; 16GB RAM; 8GB Swap; it could very well be something else, but when I used the old CLI tool months ago it seemed to go a lot smoother. This is my first time using the new CLI tool, so not sure what to expect.
@esmoyer how many ram do you have on the machine you are trying to run the cli on?
@esmoyer can you update the CLI tool to the latest version 2.0.4 to see if it still causes the issue?
You can do npm i -g @immich/cli
to update it
@alextran1502 I can confirm that the immich upload
command in --dry-run
mode is successfully executed on two folders with a large amount of images and videos in them. Issue can be closed.
Thanks for a quick fix!
@alextran1502 seems to be working well now, thanks for the quick fix!
The bug
When I tried to upload a folder with over 8k images and videos using the new CLI, I got the following error. I think the node process ran out of memory then was killed by the OS. My laptop runs Arch and has 16 GB of RAM memory. The process consumed according to the Resource Monitor, a bit over 12 GB RAM before I got the error.
Uploading a folder that contained around 1.3k images and videos using the new CLI worked.
Below is a snippet from the content of the folder and the file count:![image](https://github.com/immich-app/immich/assets/8177290/53b1cf6f-a90f-40fe-9b78-e3eb4d73411b)
The OS that Immich Server is running on
Ubuntu 22.04.2 LTS
Version of Immich Server
v1.88.1
Version of Immich Mobile App
v1.88.0 build.112
Platform with the issue
Your docker-compose.yml content
Your .env content
Reproduction steps
Additional information
Redis and database container are hosted in a common docker container for my entire homlab, that's why they are not part of the docker-compose.yml content