search

home-assistant / core

:house_with_garden: Open source home automation that puts local control and privacy first.
https://www.home-assistant.io
Apache License 2.0
71.14k stars 29.82k forks source link

Home Assistant Memory Leak #42752

Closed McGiverGim closed 3 years ago

McGiverGim commented 3 years ago

The problem

EDIT: I edited this to remove the ONVIF integration as suspicious of the problem.

There is some memory leak in ONVIF, as stated here: https://github.com/home-assistant/core/issues/42390 It was fixed in theory in 0.117.2 but it seems at least in my case this is wrong. I was asked to open a new issue, so this is.

Several hours after removing ONVIF seems it does not fix the issue, so is clear that in my case the leak is at another place.

Here is a sample after installing 0.117.2 from 0.116.: image

Environment


## Traceback/Error logs
<!--
  If you come across any trace or error logs, please provide them.
-->
Nothing.
```txt

Additional information

Stimpy68 commented 3 years ago

Upgraded again to 0.117.2 and disabled upnp (ssdp & zeroconf), Brother integration was removed yesterday. Will keep an eye on memory usage in the coming hours.

ramyi commented 3 years ago

Try creating a snapshot. It normally happens after that for me.

On Wed, 4 Nov 2020 at 7:12 pm, Stimpy68 notifications@github.com wrote:

Upgraded again to 0.117.2 and disabled upnp (ssdp & zeroconf), Brother integration was removed yesterday. Will keep an eye on memory usage in the coming hours.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/home-assistant/core/issues/42752#issuecomment-721579553, or unsubscribe https://github.com/notifications/unsubscribe-auth/AITMFIZB4FB37WRJSM4CHB3SOEEIRANCNFSM4THOP75Q .

Stimpy68 commented 3 years ago

@ramyi Did that now, now we wait ;-) Core RAM usage stayed the same, Supervisor RAM usage increased a lot, but doesn't seem to be going down.... stays around 32% usage (3GB memory allocated), Core uses 9% at this moment.

But creating a snapshot seems to create errors:

20-11-04 08:29:14 INFO (MainThread) [supervisor.snapshots] Creating new full-snapshot with slug 2ba11093 20-11-04 08:29:14 INFO (MainThread) [supervisor.snapshots] Snapshotting 2ba11093 store Add-ons 20-11-04 08:29:14 INFO (MainThread) [supervisor.addons.addon] Building snapshot for add-on a0d7b954_nodered 20-11-04 08:29:14 INFO (MainThread) [supervisor.addons.addon] Finish snapshot for addon a0d7b954_nodered 20-11-04 08:29:14 INFO (MainThread) [supervisor.addons.addon] Building snapshot for add-on core_configurator 20-11-04 08:29:14 INFO (MainThread) [supervisor.addons.addon] Finish snapshot for addon core_configurator 20-11-04 08:29:14 INFO (MainThread) [supervisor.addons.addon] Building snapshot for add-on core_ssh 20-11-04 08:29:14 INFO (MainThread) [supervisor.addons.addon] Finish snapshot for addon core_ssh 20-11-04 08:29:14 INFO (MainThread) [supervisor.addons.addon] Building snapshot for add-on core_samba 20-11-04 08:29:14 INFO (MainThread) [supervisor.addons.addon] Finish snapshot for addon core_samba 20-11-04 08:29:14 INFO (MainThread) [supervisor.addons.addon] Building snapshot for add-on a0d7b954_influxdb 20-11-04 08:29:28 INFO (MainThread) [supervisor.addons.addon] Finish snapshot for addon a0d7b954_influxdb 20-11-04 08:29:28 INFO (MainThread) [supervisor.addons.addon] Building snapshot for add-on a0d7b954_grafana 20-11-04 08:29:28 INFO (MainThread) [supervisor.addons.addon] Finish snapshot for addon a0d7b954_grafana 20-11-04 08:29:28 INFO (MainThread) [supervisor.addons.addon] Building snapshot for add-on 7ad98f9c_zigbee2mqtt 20-11-04 08:29:28 INFO (MainThread) [supervisor.addons.addon] Finish snapshot for addon 7ad98f9c_zigbee2mqtt 20-11-04 08:29:28 INFO (MainThread) [supervisor.addons.addon] Building snapshot for add-on core_mosquitto 20-11-04 08:29:28 INFO (MainThread) [supervisor.addons.addon] Finish snapshot for addon core_mosquitto 20-11-04 08:29:28 INFO (MainThread) [supervisor.addons.addon] Building snapshot for add-on a0d7b954_glances 20-11-04 08:29:28 INFO (MainThread) [supervisor.addons.addon] Finish snapshot for addon a0d7b954_glances 20-11-04 08:29:28 INFO (MainThread) [supervisor.snapshots] Snapshotting 2ba11093 store folders 20-11-04 08:29:28 INFO (SyncWorker_2) [supervisor.snapshots.snapshot] Snapshot folder ssl 20-11-04 08:29:28 INFO (SyncWorker_2) [supervisor.snapshots.snapshot] Snapshot folder ssl done 20-11-04 08:29:28 INFO (SyncWorker_3) [supervisor.snapshots.snapshot] Snapshot folder homeassistant 20-11-04 08:30:14 ERROR (MainThread) [supervisor.snapshots] A snapshot/restore process is already running 20-11-04 08:30:47 INFO (SyncWorker_3) [supervisor.snapshots.snapshot] Snapshot folder homeassistant done 20-11-04 08:30:47 INFO (SyncWorker_2) [supervisor.snapshots.snapshot] Snapshot folder media 20-11-04 08:30:47 INFO (SyncWorker_2) [supervisor.snapshots.snapshot] Snapshot folder media done 20-11-04 08:30:47 INFO (SyncWorker_4) [supervisor.snapshots.snapshot] Snapshot folder share 20-11-04 08:30:48 INFO (SyncWorker_4) [supervisor.snapshots.snapshot] Snapshot folder share done 20-11-04 08:30:48 INFO (SyncWorker_1) [supervisor.snapshots.snapshot] Snapshot folder addons/local 20-11-04 08:30:48 INFO (SyncWorker_1) [supervisor.snapshots.snapshot] Snapshot folder addons/local done 20-11-04 08:30:48 INFO (MainThread) [supervisor.snapshots] Crating full-snapshot with slug 2ba11093 completed

McGiverGim commented 3 years ago

Dump of 60 minute profiling ready, here it is: heap_profile.1604474975861335.zip

Is clear that the memory had the leak during the test: image

I don't know if some of the experts can look at it to see if it explains the problem @hunterjm @bdraco are you interested?

I'm trying to open it with the browser, but is slow, I will update this post later if I can get an image with it. EDIT: This is the image produced by the Browser: image

Now, I will test the Brother test version provided by @bieniu

Stimpy68 commented 3 years ago

1 hour passed, and no noticeable increase in RAM usage after disabling upnp

Knipsel2

Just after 9:00 (the dip) disabled upnp and upgraded from 116.4 to 117.2

DanskerUS commented 3 years ago

Creating snapshot bumps RAM usage from about 5% to 41%. Creating a second snapshot does not further increase RAM usage. Reboot restores RAM back to about 5%.

Running on a NUC w 8GB RAM.

McGiverGim commented 3 years ago

Maybe is too soon to comment, but with the updated test version of Brother by @bieniu my system seems stable in memory after two hours. image

I have a slightly more memory usage, I don't know if this will continue growing in the next hours, seems stable. I will keep an eye on it. I have noticed too that in 0.117.2 I have more CPU usage (about 6%, with 0.116.4 it was about 3%).

Others have commented that after removing Brother the problem remains, so maybe there are two different memory leaks?

It is interesting to decide if the memory dump that I did points to Brother or not. If not maybe the tool used for this memory dump is not the correct one. We need one that can help developers to know where the problem is. Maybe the pysnmp OidOrderedDict is used by Brother? I think remember it uses SNMP and OIDs.

Stimpy68 commented 3 years ago

@McGiverGim I removed the brother integration and the mem leak remained, I disabled upnp and it seems that this solved it (for now), memory usage is rising but MUCH more slowly than when upnp was enabled. But it has just been 2 hours now, so will get back after 24 hours.

bieniu commented 3 years ago

@McGiverGim Memory leakage in Brother integration can only occur when restarting the HA when the printer is turned off/offline. Is this your case?

McGiverGim commented 3 years ago

@McGiverGim Memory leakage in Brother integration can only occur when restarting the HA when the printer is turned off/offline. Is this the case for you?

Yes, it is. I have usually it powered off, and I restarted at each update, so it seems my case.

bieniu commented 3 years ago

@McGiverGim Could you do one more test for me? Save __init__.py file from this gist https://gist.github.com/bieniu/096bd2e50b1befb6b8c6bea6616a8ba6 to /config/custom_components/brother, turn off the printer, restart HA and watch the memory usage.

McGiverGim commented 3 years ago

@McGiverGim Could you do one more test for me? Save init.py file from this gist https://gist.github.com/bieniu/096bd2e50b1befb6b8c6bea6616a8ba6 to /config/custom_components/brother, turn off the printer, restart HA and watch the memory usage.

Of course, I will do it and report in one or two hours...

tr1plus commented 3 years ago

I feel it's either UPNP or Brother (my brother printer is always off when starting home assistant - I only turn it on when I need to print which is not often)

image

EDIT: Maybe not - I think my RAM started swapping - appologies

image

McGiverGim commented 3 years ago

@McGiverGim Could you do one more test for me? Save init.py file from this gist https://gist.github.com/bieniu/096bd2e50b1befb6b8c6bea6616a8ba6 to /config/custom_components/brother, turn off the printer, restart HA and watch the memory usage.

@bieniu One thing more, only to be sure... do you want that I replace the file from your test version or that I remove your test version and I put this file there, all alone?

bieniu commented 3 years ago

@McGiverGim Please only replace __init__.py file.

McGiverGim commented 3 years ago

@bieniu with the gist init file, it seems the leak is here again:

image

If I'm not wrong this is the only difference between both init files:

image

bieniu commented 3 years ago

Yes, that's the only difference. Thanks for help with testing.

hunterjm commented 3 years ago

I merged the fix for Brother and tagged it for 0.117.4 - Will keep this open since there are some reports in here of an issue without the Brother integration enabled.

rickkdotnet commented 3 years ago

I also see the memory leak since 0.117.x but I don't have the Brother integration, see below for my integrations. Will try some things in this thread.

image
DanskerUS commented 3 years ago

Updated to 117.3. Making a snapshot still cause Supervisor RAM Usage to jump to 40.6 percent. Never had ONVIF or Brother integration in use here.

hmmbob commented 3 years ago

Suggest to install 0.117.4, as it contains the brother fix:

https://github.com/home-assistant/core/releases/tag/0.117.4

DanskerUS commented 3 years ago

Will do as soon as it become available, thank you.

DanskerUS commented 3 years ago

Updated to 117.4.

Same: making a snapshot makes Supervisor RAM Usage jump to 41.4%.

DanskerUS commented 3 years ago

Rebooting drops it back to 1.5%

elupus commented 3 years ago

@DanskerUS snapshot memory usage doesn't seem related to this issue with leaking over time.

DanskerUS commented 3 years ago

@elupus OK, should I open a separate ticket?

Adminiuga commented 3 years ago

1.5% after a rebooting is not indicative. Reboot the system and run it without snapshotting it for a few days, the memory usage after a few days is going to be your baseline, e.g. 51% Take a snapshot. Does it jump to 92% and stays there? if yes, then open another ticket.

DanskerUS commented 3 years ago

After running some hours the baseline is around 5%. Soooo, should I turn off the nightly backup...?

Is it normal to see such a bump in memory use from a snapshot, did not notice this under 116.x or earlier?

Adminiuga commented 3 years ago

If you are taking snapshots at night as the backup, then no need to turn it off. Just take a screenshot of memory usage before and after the snapshot. Keep it till next night/next snapshot and take a screenshot before/after the 2nd night snapshot. Open the ticket, post the details.

Stimpy68 commented 3 years ago

24 hours later and upgrading to 0.117.3, some hours later to 0.117.4 (the dips in the chart) RAM usage stayed stable (upnp disabled)

Knipsel

Will enable upnp now and will keep an eye on RAM usage

Stimpy68 commented 3 years ago

1 hour after enabling upnp, memory usage seems stable (running 0.117.4)

Knipsel2

Turned on my Brother printer and will wait for HA to discover it, will add that.

Stimpy68 commented 3 years ago

Memory usage still stable after enabling UPNP again and installing the Brother integration. So issue seems te be fixed in 0.117.4 Maybe more people can verify this?

sz0wxc commented 3 years ago

Have a PI3 running Hassos and while I'm not seeing the large mem increase that some see I do see a slight increase.. also running Adguard add-on. after 117 upgrade things become so slow i have to reboot to get speed back.. recently I disabled adguard and went back to router for DNS cause things are so slow. moved back to 116.2 and speed seems ok.. not getting the slowness I had on 117. For a while, I thought it was just adguard but I think its hass

thijsdejong commented 3 years ago

I have the exact same problem on 0.117.4 Memory slowly fills up and swap also fills up eventually grinding everything to a dead stop. Reverting back to 0.116.4 seems like a temporary solution.

hmmbob commented 3 years ago

@sz0wxc and @thijsdejong when you say 0.117, did you try the latest release (currently 0.117.5)?

rickkdotnet commented 3 years ago

0.117.4 doesn't fix it for me, but that's expected as I don't use the Brother integration. I don't see any relevant fixes in 0.117.5. It looks like everyone in this thread uses the esphome integration, so maybe that's one to look into..

thijsdejong commented 3 years ago

@sz0wxc and @thijsdejong when you say 0.117, did you try the latest release (currently 0.117.5)?

0.117.4. But I don't see any change related to this in 0.117.5 vs .4 correct?

thijsdejong commented 3 years ago

0.117.4 doesn't fix it for me, but that's expected as I don't use the Brother integration. I don't see any relevant fixes in 0.117.5. It looks like everyone in this thread uses the esphome integration, so maybe that's one to look into..

I don't use the esphome integration

hunterjm commented 3 years ago

@rickkdotnet - I use ESP Home and am not seeing the same issues. There were also no changes to ESP Home in 0.117 other than translation string updates, so it is very unlikely to be the culprit.

EricReiche commented 3 years ago

I was hoping it's ONVIF, but after trying 0.117.4 yesterday, my automations again did not work this morning. Here is the list of my integrations.

- -
Amazon Alexa alexa
Home Assistant API api
Auth auth
Automatisierung automation
Binärsensor binary_sensor
Kalender calendar
Kamera camera
Google Cast cast
Klima climate
Home Assistant Cloud cloud
Configuration config
Coronavirus (COVID-19) coronavirus
Counter counter
Abdeckung cover
Default Config default_config
Denon AVR Network Receivers denonavr
Device Automation device_automation
Geräte-Tracker device_tracker
DoorBird doorbird
FFmpeg ffmpeg
AVM FRITZ!Box fritzbox
Home Assistant Frontend frontend
Google Calendars google
Gruppe group
HACS hacs
Logitech Harmony Hub harmony
Hass.io hassio
History history
Home Assistant homeassistant
HTTP http
Philips Hue hue
IFTTT ifttt
Image image
Boolescher Wert eingeben input_boolean
Eingabe Datum/Uhrzeit input_datetime
Numerische Eingabe input_number
Auswahlfeld input_select
Texteingabe input_text
KNX knx
Licht light
Schloss lock
Logbook logbook
Lovelace lovelace
Map map
Mediaplayer media_player
Media Source media_source
Meteorologisk institutt (Met.no) met
Mobile App mobile_app
MQTT mqtt
Benachrichtigung notify
Home Assistant Onboarding onboarding
ONVIF onvif
Persistent Notification persistent_notification
Person person
Pi-hole pi_hole
Plex Media Server plex
Sony PlayStation 4 ps4
Python Scripts python_script
Recorder recorder
Fernbedienung remote
Szene scene
Skript script
Search search
Sensor sensor
Shelly smart home shelly
Spotify spotify
Simple Service Discovery Protocol (SSDP) ssdp
Stream stream
Sonne sun
Schalter switch
Synology DSM synology_dsm
Systemzustand system_health
System Log system_log
Tags tag
Tankerkönig tankerkoenig
Timer timer
TP-Link Kasa Smart tplink
Text-to-Speech (TTS) tts
Ubiquiti UniFi unifi
Unifi Protect unifiprotect
Updater updater
UPnP upnp
Viessmann ViCare vicare
Volkswagen WeConnect volkswagencarnet
waste_collection_schedule waste_collection_schedule
Water Heater water_heater
Weather weather
Webhook webhook
Home Assistant WebSocket API websocket_api
Zero-configuration networking (zeroconf) zeroconf
Zone zone
sz0wxc commented 3 years ago

@sz0wxc and @thijsdejong when you say 0.117, did you try the latest release (currently 0.117.5)?

I had tried 117.4 but have not tried 117.5. As for addons I only have Adguard, Duck DNS, Terminal & SSH, and Chrony.

McGiverGim commented 3 years ago

As for integrations I only have Adguard, Duck DNS, Terminal & SSH, and Chrony

This are addons, not integrations. Check Configuration -> Integrations.

Gunth commented 3 years ago

I also still have the memory issue, less then before but continue to reboot 1 / 24h ...

Here is my list of integrations

  AdGuard Homeadguard Documentation Issues
  Air Qualityair_quality Documentation Issues
  AirVisualairvisual Documentation Issues
  Alarm control panelalarm_control_panel Documentation Issues
  Alertalert Documentation Issues
  Almondalmond Documentation Issues
  Home Assistant APIapi Documentation Issues
  Authauth Documentation Issues
  Automationautomation Documentation Issues
  Axisaxis Documentation Issues
  Binary sensorbinary_sensor Documentation Issues
  Blitzortungblitzortung Documentation Issues
  Brother Printerbrother Documentation Issues
  Calendarcalendar Documentation Issues
  Cameracamera Documentation Issues
  Google Castcast Documentation Issues
  Climateclimate Documentation Issues
  Configurationconfig Documentation Issues
  Conversationconversation Documentation Issues
  Daily Sensordaily Documentation Issues
  Device Automationdevice_automation Documentation Issues
  Device trackerdevice_tracker Documentation Issues
  Emulated Hueemulated_hue Documentation Issues
  Fanfan Documentation Issues
  FFmpegffmpeg Documentation Issues
  Home Assistant Frontendfrontend Documentation Issues
  Geolocationgeo_location Documentation Issues
  Google Calendarsgoogle Documentation Issues
  Groupgroup Documentation Issues
  HACShacs Documentation Issues
  Logitech Harmony Hubharmony Documentation Issues
  Hass.iohassio Documentation Issues
  Historyhistory Documentation Issues
  Home Assistanthomeassistant Documentation Issues
  HomeKit Bridgehomekit Documentation Issues
  HomeKit Controllerhomekit_controller Documentation Issues
  HTTPhttp Documentation Issues
  Philips Huehue Documentation Issues
  Imageimage Documentation Issues
  InfluxDBinfluxdb Documentation Issues
  Input booleaninput_boolean Documentation Issues
  Input numberinput_number Documentation Issues
  Input selectinput_select Documentation Issues
  Input textinput_text Documentation Issues
  Home Assistant iOSios Documentation Issues
  Internet Printing Protocol (IPP)ipp Documentation Issues
  Lightlight Documentation Issues
  Logbooklogbook Documentation Issues
  Loggerlogger Documentation Issues
  Lovelacelovelace Documentation Issues
  Mapmap Documentation Issues
  Media playermedia_player Documentation Issues
  Mobile Appmobile_app Documentation Issues
  MQTTmqtt Documentation Issues
  Netatmonetatmo Documentation Issues
  Notificationsnotify Documentation Issues
  Home Assistant Onboardingonboarding Documentation Issues
  OpenUVopenuv Documentation Issues
  OpenWeatherMapopenweathermap Documentation Issues
  iframe Panelpanel_iframe Documentation Issues
  Persistent Notificationpersistent_notification Documentation Issues
  Personperson Documentation Issues
  Plex Media Serverplex Documentation Issues
  Recorderrecorder Documentation Issues
  Remoteremote Documentation Issues
  Scenescene Documentation Issues
  Scriptscript Documentation Issues
  Searchsearch Documentation Issues
  Sensorsensor Documentation Issues
  Sonossonos Documentation Issues
  Streamstream Documentation Issues
  Sunsun Documentation Issues
  Switchswitch Documentation Issues
  Synology DSMsynology_dsm Documentation Issues
  System Healthsystem_health Documentation Issues
  System Logsystem_log Documentation Issues
  Tagtag Documentation Issues
  TP-Link Kasa Smarttplink Documentation Issues
  Text-to-Speech (TTS)tts Documentation Issues
  Weatherweather Documentation Issues
  Webhookwebhook Documentation Issues
  Home Assistant WebSocket APIwebsocket_api Documentation Issues
  Belkin WeMowemo Documentation Issues
  Yeelightyeelight Documentation Issues
  Zero-configuration networking (zeroconf)zeroconf Documentation Issues
  Zonezone Documentation Issues
JBS5 commented 3 years ago

Another HA with memory leak problems. HA will crash every 1-6 hours.

Home Assistant: 0.117.4 Supervisor: 2020.11.0 HassOS 4.15

  Amazon Alexaalexa Documentatie Problemen
  Home Assistant APIapi Documentatie Problemen
  Authauth Documentatie Problemen
  Automatiseringautomation Documentatie Problemen
  Binaire sensorbinary_sensor Documentatie Problemen
  Cameracamera Documentatie Problemen
  Google Castcast Documentatie Problemen
  Klimaatclimate Documentatie Problemen
  Home Assistant Cloudcloud Documentatie Problemen
  Configurationconfig Documentatie Problemen
  Countercounter Documentatie Problemen
  Bedekkingcover Documentatie Problemen
  deCONZdeconz Documentatie Problemen
  Default Configdefault_config Documentatie Problemen
  Device Automationdevice_automation Documentatie Problemen
  Apparaat trackerdevice_tracker Documentatie Problemen
  Discoverydiscovery Documentatie Problemen
  DSMR Slimme Meterdsmr Documentatie Problemen
  ESPHomeesphome Documentatie Problemen
  Ventilatorfan Documentatie Problemen
  FFmpegffmpeg Documentatie Problemen
  Home Assistant Frontendfrontend Documentatie Problemen
  Google Assistantgoogle_assistant Documentatie Problemen
  Groepgroup Documentatie Problemen
  HACShacs Documentatie Problemen
  Hass.iohassio Documentatie Problemen
  Historyhistory Documentatie Problemen
  Home Assistanthomeassistant Documentatie Problemen
  HomeKit Bridgehomekit Documentatie Problemen
  HTTPhttp Documentatie Problemen
  Imageimage Documentatie Problemen
  InfluxDBinfluxdb Documentatie Problemen
  Boolean invoerinput_boolean Documentatie Problemen
  Voer datum en tijd ininput_datetime Documentatie Problemen
  Numerieke invoerinput_number Documentatie Problemen
  Invoer selectieinput_select Documentatie Problemen
  Tekstinvoerinput_text Documentatie Problemen
  Home Assistant iOSios Documentatie Problemen
  Internet Printing Protocol (IPP)ipp Documentatie Problemen
  Lichtlight Documentatie Problemen
  Slotlock Documentatie Problemen
  Logbooklogbook Documentatie Problemen
  Loggerlogger Documentatie Problemen
  Lovelacelovelace Documentatie Problemen
  Mapmap Documentatie Problemen
  Mediaspelermedia_player Documentatie Problemen
  Media Sourcemedia_source Documentatie Problemen
  Mobile Appmobile_app Documentatie Problemen
  Notificeernotify Documentatie Problemen
  Home Assistant Onboardingonboarding Documentatie Problemen
  Persistent Notificationpersistent_notification Documentatie Problemen
  Persoonperson Documentatie Problemen
  Recorderrecorder Documentatie Problemen
  RFXCOM RFXtrxrfxtrx Documentatie Problemen
  Samsung Smart TVsamsungtv Documentatie Problemen
  Scènescene Documentatie Problemen
  Scriptscript Documentatie Problemen
  Searchsearch Documentatie Problemen
  Sensorsensor Documentatie Problemen
  Simple Service Discovery Protocol (SSDP)ssdp Documentatie Problemen
  Streamstream Documentatie Problemen
  Zonsun Documentatie Problemen
  Schakelaarswitch Documentatie Problemen
  Systeemstatussystem_health Documentatie Problemen
  System Logsystem_log Documentatie Problemen
  Tagstag Documentatie Problemen
  Timertimer Documentatie Problemen
  TP-Link Kasa Smarttplink Documentatie Problemen
  Text-to-Speech (TTS)tts Documentatie Problemen
  Updaterupdater Documentatie Problemen
  UPnPupnp Documentatie Problemen
  Utility Meterutility_meter Documentatie Problemen
  Stofzuigenvacuum Documentatie Problemen
  Weatherweather Documentatie Problemen
  Webhookwebhook Documentatie Problemen
  Home Assistant WebSocket APIwebsocket_api Documentatie Problemen
  Xiaomi Miioxiaomi_miio Documentatie Problemen
  Zero-configuration networking (zeroconf)zeroconf Documentatie Problemen
  Zonezone Documentatie Problemen
bdraco commented 3 years ago

Has anyone used objgraph to find the leak yet? I could probably put together a small leak detector integration if we are really stuck on this.

Gunth commented 3 years ago

To be honest I have not idea ho to use objgraph :-(

By the way, I tried with 0.117.5 but same issue

McGiverGim commented 3 years ago

Has anyone used objgraph to find the leak yet? I could probably put together a small leak detector integration if we are really stuck on this.

It will be great to have tools like this in the profiler integration. The added memory profiler is not enough?

bdraco commented 3 years ago

Has anyone used objgraph to find the leak yet? I could probably put together a small leak detector integration if we are really stuck on this.

It will be great to have tools like this in the profiler integration. The added memory profiler is not enough?

The memory profiler is great, but it doesn't provide the ability to dump objects in real time

bdraco commented 3 years ago

I've updated the profiler (https://github.com/bdraco/profiler) with two new services to watch object growth over time and dump interesting objects. I've also opened https://github.com/home-assistant/core/pull/42951 to merge the changes into core.

Example usage:

Screen Shot 2020-11-07 at 11 55 10 AM Screen Shot 2020-11-07 at 11 55 22 AM
hmmbob commented 3 years ago

I'm wondering if the issue really is within HA core, or maybe in the OS or Supervisor. As far as I can tell, all users seeing this issue are using a supervised setup in one shape or form. The few users I know using docker aren't seeing this issue (incl myself), nor did I see any venv user in this thread.