Lagging\hanging system after 115.0\115.1

to4ko commented 4 years ago

The problem

I'm running supervised version of HA since 0.86 just simply updating it with no any major issues, Once updated to 115.0 (and later to 115.1) i got lagging\hanging system (even gui not responding, were checking logs\restarting via portainer)

Looks like it starts at the moment of zeroconf\discovery start messages appears in log

Environment

Home Assistant Core release with the issue: 0.115.0 and 0.115.1
Last working Home Assistant Core release (if known): 0.114.4
Operating environment (OS/Container/Supervised/Core): Ubuntu 20.04.1\18.04.5 Supervised
Integration causing this issue:
Link to integration documentation on our website:

Problem-relevant `configuration.yaml`

were trying to get exact part of my config (https://github.com/to4ko/myconfig) with no major success - looks like it's somewhere in sensors, but kind of every time different part of my sensors.

Traceback/Error logs

2020-09-19 20:06:48 INFO (SyncWorker_29) [homeassistant.components.zeroconf] Starting Zeroconf broadcast
2020-09-19 20:07:27 ERROR (MainThread) [homeassistant.components.hassio.handler] Timeout on /discovery request
2020-09-19 20:07:27 ERROR (MainThread) [homeassistant.components.hassio.discovery] Can't read discover info: 
2020-09-19 20:07:46 WARNING (MainThread) [homeassistant.core] Something is blocking Home Assistant from wrapping up the start up phase. We're going to continue anyway. Please report the following info at http://bit.ly/2ogP58T : sensor.systemmonitor, image, hassio, timer, esphome, switch.mqtt, sensor.unifiprotect, person, search, onboarding, tts, sensor.github, proximity, light.mqtt, logbook, switch.unifi, switch.esphome, ssdp, notify.group, sensor.version, media_player.cast, binary_sensor.command_line, fan.xiaomi_miio, mqtt, camera.unifiprotect, auth, binary_sensor.mqtt, sensor.gismeteo, hacs, frontend, group, binary_sensor, start_time, binary_sensor.unifiprotect, input_datetime, zone, binary_sensor.template, binary_sensor.updater, sensor.darksky, sensor.sql, sensor.rest, camera.mjpeg, binary_sensor.esphome, device_tracker.unifi, sensor, wake_on_lan, script, media_player.mpd, persistent_notification, sensor.time_date, sensor.esphome, device_automation, binary_sensor.tod, unifiprotect, http, life360, light.esphome, input_number, telegram_bot, map, config, cast, system_health, binary_sensor.ping, utility_meter, websocket_api, sensor.start_time, upnp, counter, xiaomi_miio, light.xiaomi_miio, sensor.template, ios, notify.ios, api, media_player, homeassistant, history, logger, recorder, notify.telegram, sensor.unifi, fan, unifi, sensor.integration, sensor.min_max, sensor.statistics, ffmpeg, discovery, input_boolean, sensor.mqtt, ipp, sensor.command_line, zeroconf, lovelace, device_tracker, binary_sensor.yeelight, conversation, switch.xiaomi_miio, notify.file, automation, sensor.uptime, gismeteo, light.yeelight, sensor.utility_meter, switch.unifiprotect, updater, camera, yeelight, sensor.cpuspeed, notify, sensor.hacs, weather, homekit_controller, binary_sensor.workday, weather.darksky, sensor.nut, system_log, input_select, sun, switch, weather.gismeteo, switch.template, nut, binary_sensor.trend, sensor.ios, light
2020-09-19 20:08:37 INFO (SyncWorker_38) [mpd.base] Calling MPD disconnect()
2020-09-19 20:08:37 INFO (SyncWorker_38) [mpd.base] Calling MPD disconnect()
2020-09-19 20:08:37 INFO (MainThread) [homeassistant.core] Timer:starting
2020-09-19 20:08:37 WARNING (MainThread) [homeassistant.components.sensor] Updating gismeteo sensor took longer than the scheduled update interval 0:00:30
2020-09-19 20:08:37 WARNING (MainThread) [homeassistant.components.binary_sensor] Updating ping binary_sensor took longer than the scheduled update interval 0:00:33
2020-09-19 20:08:37 WARNING (MainThread) [homeassistant.helpers.entity] Update of binary_sensor.j3455 is taking over 10 seconds
2020-09-19 20:09:21 WARNING (MainThread) [homeassistant.components.media_player] Updating mpd media_player took longer than the scheduled update interval 0:00:10
2020-09-19 20:09:21 WARNING (MainThread) [homeassistant.components.sensor] Updating rest sensor took longer than the scheduled update interval 0:00:15
2020-09-19 20:09:22 WARNING (MainThread) [homeassistant.components.sensor] Updating rest sensor took longer than the scheduled update interval 0:00:20
2020-09-19 20:09:22 ERROR (MainThread) [homeassistant.components.nut] Timeout fetching NUT resource status data
2020-09-19 20:09:22 ERROR (MainThread) [homeassistant.components.nut] Timeout fetching NUT resource status data
2020-09-19 20:09:22 ERROR (MainThread) [homeassistant.components.nut] Timeout fetching NUT resource status data
2020-09-19 20:09:22 ERROR (MainThread) [homeassistant.components.nut] Timeout fetching NUT resource status data
2020-09-19 20:09:24 INFO (SyncWorker_11) [homeassistant.components.proximity] life360 Vova Tsymbal: proximity calculation complete
2020-09-19 20:10:34 WARNING (MainThread) [homeassistant.components.weather] Updating darksky weather took longer than the scheduled update interval 0:00:30
2020-09-19 20:10:36 WARNING (MainThread) [homeassistant.components.binary_sensor] Updating ping binary_sensor took longer than the scheduled update interval 0:00:29
2020-09-19 20:10:36 WARNING (MainThread) [homeassistant.components.switch] Updating xiaomi_miio switch took longer than the scheduled update interval 0:00:30
2020-09-19 20:10:36 WARNING (MainThread) [homeassistant.components.light] Updating xiaomi_miio light took longer than the scheduled update interval 0:00:30
2020-09-19 20:10:36 WARNING (MainThread) [homeassistant.components.fan] Updating xiaomi_miio fan took longer than the scheduled update interval 0:00:30
2020-09-19 20:10:36 WARNING (MainThread) [homeassistant.components.weather] Updating gismeteo weather took longer than the scheduled update interval 0:00:30
2020-09-19 20:10:36 WARNING (MainThread) [homeassistant.components.sensor] Updating rest sensor took longer than the scheduled update interval 0:00:30
2020-09-19 20:10:36 WARNING (MainThread) [homeassistant.components.sensor] Updating cpuspeed sensor took longer than the scheduled update interval 0:00:30
2020-09-19 20:10:36 WARNING (MainThread) [homeassistant.components.sensor] Updating systemmonitor sensor took longer than the scheduled update interval 0:00:30
2020-09-19 20:10:36 WARNING (MainThread) [homeassistant.components.binary_sensor] Updating ping binary_sensor took longer than the scheduled update interval 0:00:31
2020-09-19 20:10:41 WARNING (MainThread) [homeassistant.components.sensor] Updating gismeteo sensor took longer than the scheduled update interval 0:00:30
2020-09-19 20:10:41 WARNING (MainThread) [homeassistant.components.binary_sensor] Updating ping binary_sensor took longer than the scheduled update interval 0:00:32
2020-09-19 20:10:41 WARNING (MainThread) [homeassistant.components.binary_sensor] Updating ping binary_sensor took longer than the scheduled update interval 0:00:33
2020-09-19 20:10:42 WARNING (MainThread) [homeassistant.components.binary_sensor] Updating command_line binary_sensor took longer than the scheduled update interval 0:01:00
2020-09-19 20:10:42 WARNING (MainThread) [homeassistant.components.binary_sensor] Updating ping binary_sensor took longer than the scheduled update interval 0:01:00
2020-09-19 20:10:42 WARNING (MainThread) [homeassistant.components.sensor] Updating rest sensor took longer than the scheduled update interval 0:01:00
2020-09-19 20:10:42 WARNING (MainThread) [homeassistant.components.sensor] Updating command_line sensor took longer than the scheduled update interval 0:01:00
2020-09-19 20:10:42 WARNING (MainThread) [homeassistant.helpers.entity] Update of sensor.main_input_boolean_sasha_tracker is taking over 10 seconds
2020-09-19 20:10:42 WARNING (MainThread) [homeassistant.helpers.entity] Update of sensor.main_state is taking over 10 seconds
2020-09-19 20:10:42 WARNING (MainThread) [homeassistant.helpers.entity] Update of binary_sensor.nuc is taking over 10 seconds
2020-09-19 20:10:42 WARNING (MainThread) [homeassistant.helpers.entity] Update of switch.power_strip_s is taking over 10 seconds
2020-09-19 20:10:42 WARNING (MainThread) [homeassistant.helpers.entity] Update of switch.gateway_plug_entrance_hb_usb is taking over 10 seconds
2020-09-19 20:10:42 WARNING (MainThread) [homeassistant.helpers.entity] Update of switch.gateway_plug_entrance_hb is taking over 10 seconds
2020-09-19 20:10:42 WARNING (MainThread) [homeassistant.helpers.entity] Update of light.philips_e27_s is taking over 10 seconds
2020-09-19 20:10:42 WARNING (MainThread) [homeassistant.helpers.entity] Update of light.mirror_light_b is taking over 10 seconds
2020-09-19 20:10:42 WARNING (MainThread) [homeassistant.helpers.entity] Update of light.philips_e27_mb is taking over 10 seconds
2020-09-19 20:10:42 WARNING (MainThread) [homeassistant.helpers.entity] Update of fan.purifier2s is taking over 10 seconds
2020-09-19 20:10:42 WARNING (MainThread) [homeassistant.helpers.entity] Update of fan.humidifier2 is taking over 10 seconds
2020-09-19 20:10:42 WARNING (MainThread) [homeassistant.helpers.entity] Update of binary_sensor.usg_gateway is taking over 10 seconds
2020-09-19 20:10:42 WARNING (MainThread) [homeassistant.helpers.entity] Update of binary_sensor.google_dns is taking over 10 seconds
2020-09-19 20:10:42 WARNING (MainThread) [homeassistant.helpers.entity] Update of binary_sensor.h96_v is taking over 10 seconds
2020-09-19 20:10:42 WARNING (MainThread) [homeassistant.helpers.entity] Update of binary_sensor.tv_box is taking over 10 seconds
2020-09-19 20:10:42 WARNING (MainThread) [homeassistant.helpers.entity] Update of sensor.config_line_count_sh_code is taking over 10 seconds
2020-09-19 20:10:42 WARNING (MainThread) [homeassistant.helpers.entity] Update of binary_sensor.rpi is taking over 10 seconds
2020-09-19 20:10:42 WARNING (MainThread) [homeassistant.helpers.entity] Update of sensor.zigbee2mqtt_addon is taking over 10 seconds
2020-09-19 20:10:42 WARNING (MainThread) [homeassistant.helpers.entity] Update of sensor.supervisor_updates is taking over 10 seconds
2020-09-19 20:10:42 WARNING (MainThread) [homeassistant.helpers.entity] Update of sensor.set_utility_from_main is taking over 10 seconds
2020-09-19 20:10:42 WARNING (MainThread) [homeassistant.helpers.entity] Update of sensor.cpu_speed is taking over 10 seconds
2020-09-19 20:10:42 WARNING (MainThread) [homeassistant.helpers.entity] Update of sensor.gismeteo_temperature is taking over 10 seconds

Additional information

bdraco commented 4 years ago

Please post a py-spy dump, top, and record.

https://github.com/benfred/py-spy/releases

to4ko commented 4 years ago

Please post a py-spy dump, top, and record.

https://github.com/benfred/py-spy/releases

Could you please advice exact commands. Also - this should be run on HA restart?

bdraco commented 4 years ago

https://community.home-assistant.io/t/python3-high-cpu-usage/160012/37?u=bdraco

to4ko commented 4 years ago

https://community.home-assistant.io/t/python3-high-cpu-usage/160012/37?u=bdraco

will do it this night.

digging little bit deeper - looks like this one (i think it's one of them) causing issues

  - platform: rest
    resource: http://192.168.1.30:8123/api/hassio/addons/7ad98f9c_zigbee2mqtt/info
    name: zigbee2mqtt_addon
    value_template: '{{ value_json.data.state }}'
    scan_interval: 60
    headers:
      Authorization: !secret mqtt_long_token
      Content-Type: application/json
    json_attributes_path: "$.data"
    json_attributes:
      - version
      - version_latest

as well as this one

  - platform: mqtt
    name: Zigbee2mqtt Bridge Uptime
    state_topic: "zigbee2mqtt_start_at"
    value_template: '{{ as_timestamp(now()) - value | int }}'
    unit_of_measurement: 's'
    force_update: true
    icon: mdi:av-timer

to4ko commented 4 years ago

closing this one as it's related to myself being dumb with templates. thanks!

home-assistant / core