search

home-assistant / core

:house_with_garden: Open source home automation that puts local control and privacy first.
https://www.home-assistant.io
Apache License 2.0
71.23k stars 29.85k forks source link

Inappropriate memory allocation #124457

Open thesmurfito opened 3 weeks ago

thesmurfito commented 3 weeks ago

The problem

I have a large setup with many(!) automations running every 15 minutes. I noticed something reminiscent of a memory leak, where RAM consumption would be around 10% at boot up, then every run of automations it would increase by around 5%, until the system would encounter out of memory and HA core would reboot.

after alot of time with profiler it seemed the issue wa related to the traces in automations (the count was high).

Reducing traces to only 4, immediately alleviated the issue. Consumption starts at 10%, goes up to 20% during the first runs of automations then remains steady.

After increasing the traces to 10, consumption quickly goes to 35% and remains steady at that level.

I suspect that all the previous traces are stored and remain in memory, so at any given trace number, it will reach steady state once the number of traces have been reached.

The screenshot shows memory usage at boot up, after a a few automations with low trace count, and then when trace count is increased.

Unfortunately I don't have the graph when it would reach 100% and crash since this wasn't tracked by my recorder previously.

1: The traces should probably not take up that much space and 2: They should probably not be stored in memory.

What version of Home Assistant Core has the issue?

2024.8.2

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant Supervised

Integration causing the issue

No response

Link to integration documentation on our website

No response

Diagnostics information

Screenshot 2024-08-22 at 12 17 28

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

System setup is 64 GB Ram around 100 automations running every 15 minutes. HA core would crash every 3-4 hours due to out-of-memory, at which point ram consumption would briefly normalise then increase again. The memory graph was identical to any memory-leak type situation 

### Tasks
bh commented 2 weeks ago

Yes, same here. I need to restart the HA Core Docker Container every 6h on my PI4 with 8GB RAM. (2024.8.2)

thesmurfito commented 2 weeks ago

Yes, same here. I need to restart the HA Core Docker Container every 6h on my PI4 with 8GB RAM. (2024.8.2)

Hey. Did you try reducing the stored_traces on your automations and see if that helps ?