rommapp / romm

A beautiful, powerful, self-hosted rom manager
https://romm.app
GNU Affero General Public License v3.0
1.77k stars 80 forks source link

[Bug] High CPU idle usage #801

Closed ThibaultNocchi closed 4 months ago

ThibaultNocchi commented 5 months ago

RomM version 3.0.3

Describe the bug When not using it and when no scheduled job is running, Romm idles around 10% usage of my CPU. For instance, Photoprism hovers at 8% when scanning the photos directory hourly.

My CPU is an i3 12100.

To Reproduce Use top or anything else to monitor CPU usage

Expected behavior At best it shouldn't consume all that CPU when idling. Maybe we could reduce in config what's taking up power. Or if nothing is possible, I'm just curious to know why

Screenshots image

Additional context I'm running the official container image. When running htop it seems that the culprit process is python3 scheduler.py

As workaround I've limited the CPU the container can use to 250m of a vCPU instead of the ~750m it was actually using

Thanks!

gantoine commented 5 months ago

That's so interesting, I've been investigating high RAM usage (~500MB at idle) for a bit with no luck, but high CPU usage is not something we've observed. I'll have a look when I find a bit of time.

When running htop it seems that the culprit process is python3 scheduler.py

This will be helpful, thank you! Could you also post your docker-compose?

As workaround I've limited the CPU the container can use to 250m of a vCPU instead of the ~750m it was actually using

This is good practice, I do the same with all my containers.

ThibaultNocchi commented 5 months ago

I'm not running in Docker but in Kube, but here is the equivalent Docker:

docker run \
 -e DB_HOST=romm-mariadb -e DB_NAME=romm -e DB_USER=romm -e ROMM_AUTH_USERNAME=admin -e IGDB_CLIENT_ID=secret -e IGDB_CLIENT_SECRET=secret -e DB_PASSWD=secret -e ROMM_AUTH_SECRET_KEY=secret -e ROMM_AUTH_PASSWORD=secret \
 --cpus=0.25 \
 rommapp/romm:3.0.3

With volumes and a config.yml mounted. I also have MariaDB in another pod.

I'll happily provide you anything you need. :)

ThibaultNocchi commented 4 months ago

Hey!

Yesterday I noticed a scanning bug, and guess I didn't read the Romm logs: I had multiple Redis errors saying it didn't have rights to write, such as this: https://github.com/rommapp/romm/issues/812#issuecomment-2054122396

I fixed the error by chowning it, and it seems to have fixed my CPU usage. Ofc I dropped my Prometheus database this morning so I can't verify if the fix matched the CPU usage, but now it's idling well.

Thanks for your help, we may close the issue!

gantoine commented 4 months ago

Ok awesome! This would have fixed itself when 3.1 went live and you upgraded, but I'm glad to hear CHOWNing it worked. 👍🏼