I can not pin down the exact build, but I think it started with version 1.100.0.
Every night, starting at 00:00, immich_microservices starts to consume 100% CPU of one core and redis-server starts to consume about 50% of a core of my server. This is how it looks the next morning when looking at htop:
Checking the output of docker compose logs, I can see this:
And in my monitoring I can see clearly that it all started at 00:00:
I assume this has something to do with my external library. That one sits on a NAS that shuts down every night at 23:30 and can stay off for days. I know this is not optimal for immich, but I made sure that all processes in immich (automatic rescan of librarys and what not) are off. Also, this was not an issue before version 1.100.x.
I have seen that there is an automatic library rescan job in immich that starts at 00:00, but I made sure that it is still off after updating to version 1.100.x. Currently I do not see any configuration or job that starts at 00:00.
I am pretty sure it has something to do with my external library being offline, but I can not figure out (also by reading through the changelogs) what could have changed in that regard to cause this issue.
Before 1.100.x, it was no problem shutting down the NAS. Immich was still running fine and users were able to upload their photos, without immich and redis consuming so much CPU cycles.
UPLOAD_LOCATION=/opt/immich/library
IMMICH_VERSION=release
TYPESENSE_API_KEY=***redacted***
DB_PASSWORD=***redacted***
# The values below this line do not need to be changed
###################################################################################
DB_HOSTNAME=immich-postgres
DB_USERNAME=postgres
DB_DATABASE_NAME=immich
REDIS_HOSTNAME=immich-redis
# Self-Defined
TZ=Europe/Berlin
PUID=1000
PGID=1000
Reproduction steps
1. NAS with external Library is on
2. Start immich stack
3. Shut NAS with external Library off
4. At 00:00 CPU utilization will go up to 100% for immich_microservices and to 50% for redis-server.
The bug
I can not pin down the exact build, but I think it started with version 1.100.0. Every night, starting at 00:00, immich_microservices starts to consume 100% CPU of one core and redis-server starts to consume about 50% of a core of my server. This is how it looks the next morning when looking at htop:![image](https://github.com/immich-app/immich/assets/111387593/cc9de0f2-1c19-4de5-8759-9a042a1440d8)
Checking the output of docker compose logs, I can see this:![immich_high_cpu_usage_2](https://github.com/immich-app/immich/assets/111387593/066ff570-9a3d-4f08-a3a3-715897bed821)
And in my monitoring I can see clearly that it all started at 00:00:![image](https://github.com/immich-app/immich/assets/111387593/5e2e3094-57f2-416d-b3e6-1c856651611a)
I assume this has something to do with my external library. That one sits on a NAS that shuts down every night at 23:30 and can stay off for days. I know this is not optimal for immich, but I made sure that all processes in immich (automatic rescan of librarys and what not) are off. Also, this was not an issue before version 1.100.x. I have seen that there is an automatic library rescan job in immich that starts at 00:00, but I made sure that it is still off after updating to version 1.100.x. Currently I do not see any configuration or job that starts at 00:00. I am pretty sure it has something to do with my external library being offline, but I can not figure out (also by reading through the changelogs) what could have changed in that regard to cause this issue.
Before 1.100.x, it was no problem shutting down the NAS. Immich was still running fine and users were able to upload their photos, without immich and redis consuming so much CPU cycles.
The OS that Immich Server is running on
Ubuntu Server 22.04.4 LT
Version of Immich Server
v1.103.1
Version of Immich Mobile App
n/a
Platform with the issue
Your docker-compose.yml content
Your .env content
Reproduction steps
Relevant log output
No response
Additional information
No response