search

home-assistant / core

:house_with_garden: Open source home automation that puts local control and privacy first.
https://www.home-assistant.io
Apache License 2.0
73.22k stars 30.58k forks source link

Out of Memory: Killed Process total-vm python3 #93556

Closed jryan82 closed 11 months ago

jryan82 commented 1 year ago

The problem

Hello, my homeassistant server keeps crashing. This issue has been longstanding and i've disabled most all of my extra services on my server.

What version of Home Assistant Core has the issue?

core-2023.5.4

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

No response

Link to integration documentation on our website

No response

Diagnostics information

I'm not sure which integration to pull from. Here are the logs from core below:

Home Assistant Core Error fetching 192.168.177.49 data: Unable to query the device 192.168.177.49: 2:42:04 PM – (ERROR) TP-Link Kasa Smart - message first occurred at 11:39:12 AM and shows up 53 times Unable to connect to 192.168.177.16 2:41:42 PM – (ERROR) /usr/local/lib/python3.10/site-packages/roombapy/remote_client.py - message first occurred at 11:33:12 AM and shows up 130 times Can't connect to 192.168.177.16, error: [Errno 113] Host is unreachable 2:41:42 PM – (ERROR) /usr/local/lib/python3.10/site-packages/roombapy/remote_client.py - message first occurred at 11:33:05 AM and shows up 390 times http://homeassistant.local:8123/hacsfiles/fan-control-entity-row/fan-control-entity-row.js?hacstag=19166315023:9:28 Uncaught ReferenceError: Polymer is not defined 2:40:28 PM – (ERROR) components/system_log/init.py - message first occurred at 2:37:41 PM and shows up 3 times Config entry 'Z-Wave JS' for zwave_js integration not ready yet: Failed to connect: Cannot connect to host core-zwave-js:3000 ssl:default [Connect call failed ('172.30.33.0', 3000)]; Retrying in background 2:37:59 PM – (WARNING) config_entries.py [140146531976096] Error updating Samba share: Can't install homeassistant/amd64-addon-samba:10.0.2: 500 Server Error for http+docker://localhost/v1.42/images/create?tag=10.0.2&fromImage=homeassistant%2Famd64-addon-samba&platform=linux%2Famd64: Internal Server Error ("Head "https://registry-1.docker.io/v2/homeassistant/amd64-addon-samba/manifests/10.0.2": Get "https://auth.docker.io/token?scope=repository%3Ahomeassistant%2Famd64-addon-samba%3Apull&service=registry.docker.io": EOF") 2:37:52 PM – (ERROR) Home Assistant WebSocket API Error requesting NWS forecast hourly station KSRQ data: 500, message='Internal Server Error', url=URL('https://api.weather.gov/gridpoints/TBW/79,74/forecast/hourly') 1:53:04 PM – (ERROR) National Weather Service (NWS) Unable to find referenced entities switch.garage_overheads or it is/they are currently not available 1:49:04 PM – (WARNING) helpers/service.py - message first occurred at 1:12:36 PM and shows up 6 times Timeout fetching yamaha_musiccast data 1:22:09 PM – (ERROR) MusicCast Failed to connect to websocket: 502, message='Invalid response status', url=URL('wss://socketlink.prd.aser.simplisafe.com') 12:30:36 PM – (ERROR) SimpliSafe [0x2D75:3:0x0b04]: async_initialize: all attempts have failed: [DeliveryError('Failed to deliver message: <EmberStatus.DELIVERY_FAILED: 102>'), DeliveryError('Failed to deliver message: <EmberStatus.DELIVERY_FAILED: 102>'), DeliveryError('Failed to deliver message: <EmberStatus.DELIVERY_FAILED: 102>'), DeliveryError('Failed to deliver message: <EmberStatus.DELIVERY_FAILED: 102>')] 11:33:37 AM – (WARNING) Zigbee Home Automation - message first occurred at 11:33:30 AM and shows up 31 times Error doing job: Task exception was never retrieved 11:33:26 AM – (ERROR) components/dlna_dmr/config_flow.py TemplateError('ValueError: Template error: float got invalid input 'unknown' when rendering template '{{ states('sensor.hvac_main_intake')|float - states('sensor.hvac_main_plenum')|float }}' but no default was specified') while processing template 'Template<template=({{ states('sensor.hvac_main_intake')|float - states('sensor.hvac_main_plenum')|float }}) renders=4>' for attribute '_attr_native_value' in entity 'sensor.temperature_difference_main_hvac' 11:33:26 AM – (ERROR) helpers/template_entity.py - message first occurred at 11:33:26 AM and shows up 2 times Error while processing template: Template<template=({{ states('sensor.hvac_main_intake')|float - states('sensor.hvac_main_plenum')|float }}) renders=2> 11:33:26 AM – (ERROR) helpers/template.py - message first occurred at 11:33:26 AM and shows up 2 times Config entry 'AS-AFTMM[AirPlay]' for apple_tv integration not ready yet: Not found at 192.168.177.252, waiting for discovery; Retrying in background 11:33:12 AM – (WARNING) config_entries.py - message first occurred at 11:32:59 AM and shows up 5 times Platform tplink does not generate unique IDs. ID E4:C3:2A:39:1A:57 already exists - ignoring switch.toilet_exhaust 11:33:11 AM – (ERROR) Switch - message first occurred at 11:33:11 AM and shows up 2 times The entity definition format under template: differs from the platform configuration format. See https://www.home-assistant.io/integrations/template#configuration-for-trigger-based-template-sensors 11:33:03 AM – (WARNING) Template Failed to connect to MQTT server due to exception: [Errno 111] Connection refused 11:33:02 AM – (ERROR) MQTT Config entry 'Shapes 6B24' for nanoleaf integration could not authenticate 11:32:59 AM – (WARNING) config_entries.py Ended unfinished session (id=566 from 2023-05-24 15:17:07.933775) 11:32:51 AM – (WARNING) Recorder The system could not validate that the sqlite3 database at //config/home-assistant_v2.db was shutdown cleanly 11:32:51 AM – (WARNING) Recorder

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

home-assistant[bot] commented 1 year ago

Hey there @phracturedblue, @tetienne, @home-assistant/core, mind taking a look at this issue as it has been labeled with an integration (template) you are listed as a code owner for? Thanks!

Code owner commands Code owners of `template` can trigger bot actions by commenting: - `@home-assistant close` Closes the issue. - `@home-assistant rename Awesome new title` Renames the issue. - `@home-assistant reopen` Reopen the issue. - `@home-assistant unassign template` Removes the current integration label and assignees on the issue, add the integration domain after the command.

(message by CodeOwnersMention)

template documentation template source (message by IssueLinks)

MartinHjelmare commented 1 year ago

How does the memory use look? We have the system monitor integration to help with that.

https://www.home-assistant.io/integrations/systemmonitor/

We also have the profiler integration but its memory profiling service currently doesn't support Python 3.11 although that should be fixed in the next release.

When you have some data of the memory use, you can start by disabling all custom integrations and see how the memory use changes. Then continue and disable integrations.

jryan82 commented 1 year ago

Ok, with that next release I should be able to narrow it down. I am running a lot of integrations, and don't want to start shutting them off. I currently have 4gb allocated and it's using about 3gb. I also have the system monitor installed.

On Fri, May 26, 2023 at 1:25 AM Martin Hjelmare @.***> wrote:

How does the memory use look? We have the system monitor integration to help with that.

https://www.home-assistant.io/integrations/systemmonitor/

We also have the profiler integration but its memory profiling service currently doesn't support Python 3.11 although that should be fixed in the next release.

When you have some data of the memory use, you can start by disabling all custom integrations and see how the memory use changes. Then continue and disable integrations.

— Reply to this email directly, view it on GitHub https://github.com/home-assistant/core/issues/93556#issuecomment-1563827828, or unsubscribe https://github.com/notifications/unsubscribe-auth/AITMMSFXCWMIL2LHS2KOLY3XIA5DBANCNFSM6AAAAAAYPHQO3I . You are receiving this because you authored the thread.Message ID: @.***>

litinoveweedle commented 1 year ago

Hello Martin,

maybe my OoM #93713 could be related?

MartinHjelmare commented 1 year ago

We won't know until we find the memory leak. Best to try and use the profiler integration when it's available in the next release.

litinoveweedle commented 1 year ago

I did, I was able to run profiler (supervised) output is attached in #93713

bdraco commented 1 year ago

Can you get a profiler.start_log_objects as done in https://github.com/home-assistant/core/issues/93713 as well as a callgrind file from profiler.start?

jryan82 commented 1 year ago

I'm still having the issue and have upgraded to version 2023.8.1.
My system memory is sitting at 23% utilized.
Now that Profiler is available, what would be most helpful to identify the cause? callgrind.out.1 [profile.1691609102846497.cprof.zip](https://github.com/home-assistant/core/files/12306268/profile.1691609102846497.cprof.zip) 691609102846497.zip

issue-triage-workflows[bot] commented 11 months ago

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍 This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.