Closed nazar-pc closed 2 days ago
profiler.start
servicepy-spy
from https://community.home-assistant.io/t/instructions-to-install-py-spy-on-haos/480473. (be sure to zip them up before posting as github mutates svg files)Settings -> System -> Logs -> Download
) from the https://www.home-assistant.io/integrations/profiler/ profiler.start_log_objects
service with the default settingsHey there @bdraco, mind taking a look at this issue as it has been labeled with an integration (profiler
) you are listed as a code owner for? Thanks!
(message by CodeOwnersMention)
profiler documentation profiler source (message by IssueLinks)
There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍 This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.
I'm not sure it was clear from my initial message, but this happens suddenly and very rarely. I can't simply start profiler and wait for it to happen :shrug:
I'm not sure it was clear from my initial message, but this happens suddenly and very rarely. I can't simply start profiler and wait for it to happen 🤷
Sadly, that's the worst kind of problem to track down. You could automate the monitoring of memory usage and start the profiler with an automation that watches the system monitor. You may get lucky and catch it.
I do have Prometheus client running on HA instance if that is what you mean, though not sure how would I automatically start profiler based on those metrics.
I guess what I'm looking for is more or less exact steps what to do and what kind of information to save in case it happens again in the future. It usually continues to work with high memory usage for some time before crashing and restarting.
Wouldn't say this is a massive issue for me, just hoping to collect useful information for maintainers since something somewhere does go wrong in those cases.
Just happened again, I had to reboot it before it crashed. Here are the logs before reboot, I have not seen anything particularly interesting: home-assistant_2024-08-16T14-29-11.733Z.log
There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍 This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.
The problem
Originally reported in https://github.com/home-assistant/operating-system/issues/3483
I believe this is the third time this happened to me. Home Assistant is usually running fine for days with stable memory usage and then suddenly memory usage is growing significantly. Once it hanged the whole thing, two more times (including today) just resulted in high memory usage and monitoring alerts:![Знімок екрана з 2024-07-22 12-57-18](https://github.com/user-attachments/assets/bd44020c-ccae-4170-b918-6025f7bc7130)
I looked inside and
python3 -m homeassistant --config /config
is the process that is currently using 56.6% of RAM (on VM with just 2G of RAM).Memory usage started to recover for some reason, have not done anything to cause that :man_shrugging: Home Assistant is not reachable from public Internet.
There is no records in Home Assistant Core logs. Supervisor logs have this, looks like it restarted today, though no idea why it would do that (doesn't correlate with the memory usage increase either):
Nothing interesting in any other logs.
What version of Home Assistant Core has the issue?
2024.7.0
What was the last working version of Home Assistant Core?
No response
What type of installation are you running?
Home Assistant OS
Integration causing the issue
No response
Link to integration documentation on our website
No response
Diagnostics information
No response
Example YAML snippet
No response
Anything in the logs that might be useful for us?
No response
Additional information
Eventually it crashed Home Assistant, things restarted and memory usage is back to normal:![Знімок екрана з 2024-07-22 21-03-03](https://github.com/user-attachments/assets/878628c4-fa0d-47ab-80c7-b78ae74dce34)