Daily crashes preceeded by lots of "Update of xxx is taking over xxx" for lots of components in 0.84.6

scstraus commented 5 years ago

Home Assistant release with the issue:

0.84.6

Last working Home Assistant release (if known): 0.83.2 was last version I was on without this problem

Operating environment (Hass.io/Docker/Windows/etc.):

Hassbian on Pi B3+ Component/platform:

The main hass process is disappearing. Almost all components are reporting long update times before the crash such as weather.dark_sky, media_player.itunes, calendar.google, weather.openweathermap, etc..

Description of problem:

Hass is crashing. I'm not sure why, it seems like it may be happening when I try to access the logbook from iOS (which appears to not be able to finish retrieving any items or takes so long that I try to go elsewhere in the UI), but there is some delay there, it's definitely not immediate.

Problem-relevant configuration.yaml entries and (fill out even if it seems unimportant):

homeassistant:

  name: Home
  latitude: !secret home_lat
  longitude: !secret home_long
  elevation: 260
  unit_system: metric
  time_zone: Europe/Prague
  customize: !include customize.yaml

lovelace:

  mode: yaml

frontend:

  themes: !include themes.yaml

config:

http:

  api_password: !secret hass_pass
  ssl_certificate: !secret cer_dir
  ssl_key: !secret key_dir
  base_url: !secret my_host:!secret my_port

updater:

# include_used_components: true

# discovery:

conversation:

history:

logbook:

map:

sun:

tts:

  - platform: google

logger:

  default: debug
  logs:
    homeassistant.components.mqtt: warn
    homeassistant.components.websocket_api: warn
#    homeassistant.components.nest: debug
#    homeassistant.components.sensor.nest: debug
#    sseclient: debug
#    homeassistant.components.camera: debug
#    homeassistant.components.sensor.imap_email_content: debug
#    homeassistant.components.imap_email_content: debug
#    homeassistant.components.sensor: debug
#cloud:

group: !include groups.yaml

automation: !include automations.yaml

script: !include scripts.yaml

custom_updater:

  card_urls:
    - https://raw.githubusercontent.com/rdehuyss/homeassistant-lovelace-google-calendar-card/master/custom_updater.json
    - https://raw.githubusercontent.com/kalkih/mini-media-player/master/tracker.json
    - https://raw.githubusercontent.com/kuuji/button-card/master/tracker.json

sensor:

  - platform: airvisual
    api_key: !secret airvisual_key
    monitored_conditions:
      - us

  - platform: darksky
    api_key: !secret darksky_key
    update_interval:
      minutes: 120
    forecast:
    - 0
    monitored_conditions:
    - temperature_high
    - temperature_low
    - apparent_temperature_high
    - apparent_temperature_low
    - summary
    - icon
    - nearest_storm_distance
    - precip_type
    - precip_intensity
    - precip_probability
    - temperature
    - apparent_temperature
    - dew_point
    - wind_speed
    - wind_bearing
    - wind_gust
    - cloud_cover
    - humidity
    - pressure
    - visibility
    - ozone
    - minutely_summary
    - hourly_summary
    - daily_summary

  - platform: imap_email_content
    server: imap.gmail.com
    name: cp_packages_coming_today
    port: 993
    username: !secret gmail_uname
    password: !secret hass_pass
    senders:
    - !secret my_email
    - ceskaposta@cpost.cz
    - !secret wife_email
    value_template: >-
      {% if 'Informace o Vaší zásilce' in subject %}
        delivery
      {% elif 'reset state of package sensor' in subject %}
        no_delivery
      {% endif %}

  - platform: imap_email_content
    server: imap.gmail.com
    name: house_electricity
    port: 993
    username: !secret gmail_uname
    password: !secret hass_pass
    senders:
    - !secret my_email
    - no-reply@smartconnect.apc.com
    value_template: >-
      {% if 'UPS On Battery' in subject %}
        power_out
      {% elif 'Power Restored' in subject %}
        power_on
      {% endif %}

#  - platform: gtfs
#    name: transport
#    origin: U49N198
#    destination: U484Z2
#    data: jrdata

  - platform: template
    sensors:
      bike_day:
        friendly_name: "Bike Day"
        value_template: >-
          {% if states('sensor.dark_sky_precip_probability')|float < 40 
             and states('sensor.dark_sky_daytime_high_temperature')|float < 25
             and states('sensor.dark_sky_temperature')|float > 15 
             and states('sensor.dark_sky_wind_gust')|float < 15 %}
            yes
          {% else %}
            no
          {% endif %}

  - platform: template
    sensors:
      object_detection_back:
        friendly_name: "Back Yard"
        value_template: >-
          {% if 'person' in state_attr('image_processing.tensorflow_back', 'summary') %}
            Person Detected
          {% elif 'dog' in state_attr('image_processing.tensorflow_back', 'summary') %}
            Dog Detected
          {% elif 'cat' in state_attr('image_processing.tensorflow_back', 'summary') %}
            Cat Detected
          {% else %}
            Nothing Detected
          {% endif %}

  - platform: template
    sensors:
      object_detection_front:
        friendly_name: "Front Gate"
        value_template: >-
          {% if 'person' in state_attr('image_processing.tensorflow_front', 'summary') %}
            Person Detected
          {% elif 'dog' in state_attr('image_processing.tensorflow_front', 'summary') %}
            Dog Detected
          {% elif 'cat' in state_attr('image_processing.tensorflow_front', 'summary') %}
            Cat Detected
          {% else %}
            Nothing Detected
          {% endif %}

  - platform: template
    sensors:
      object_detection_left:
        friendly_name: "Left of House"
        value_template: >-
          {% if 'person' in state_attr('image_processing.tensorflow_left', 'summary') %}
            Person Detected
          {% elif 'dog' in state_attr('image_processing.tensorflow_left', 'summary') %}
            Dog Detected
          {% elif 'cat' in state_attr('image_processing.tensorflow_left', 'summary') %}
            Cat Detected
          {% else %}
            Nothing Detected
          {% endif %}

  - platform: template
    sensors:
      object_detection_right:
        friendly_name: "Right of House"
        value_template: >-
          {% if 'person' in state_attr('image_processing.tensorflow_right', 'summary') %}
            Person Detected
          {% elif 'dog' in state_attr('image_processing.tensorflow_right', 'summary') %}
            Dog Detected
          {% elif 'cat' in state_attr('image_processing.tensorflow_right', 'summary') %}
            Cat Detected
          {% else %}
            Nothing Detected
          {% endif %}

  - platform: template
    sensors:
      tensorflow_summary_right:
        friendly_name: "Tensorflow Summary Right"
        value_template: "{{ state_attr('image_processing.tensorflow_right', 'summary') }}"

  - platform: template
    sensors:
      tensorflow_summary_left:
        friendly_name: "Tensorflow Summary Left"
        value_template: "{{ state_attr('image_processing.tensorflow_left', 'summary') }}"

  - platform: template
    sensors:
      tensorflow_summary_front:
        friendly_name: "Tensorflow Summary Front"
        value_template: "{{ state_attr('image_processing.tensorflow_front', 'summary') }}"

  - platform: template
    sensors:
      tensorflow_summary_back:
        friendly_name: "Tensorflow Summary Back"
        value_template: "{{ state_attr('image_processing.tensorflow_back', 'summary') }}"

  - platform: speedtest
    monitored_conditions:
      - ping
      - download
      - upload

#  - platform: synologydsm
#    host: !secret synology_ip
#    port: 5000
#    ssl: false
#    username: !secret syn_uname
#    password: !secret hass_pass
#    monitored_conditions:
#      - memory_real_usage
#      - network_up
#      - disk_smart_status
#      - disk_status
#      - disk_exceed_bad_sector_thr
#      - disk_below_remain_life_thr
#      - volume_percentage_used
#      - volume_disk_temp_avg
#      - volume_disk_temp_max
#  - platform: gtfs_realtime
#    trip_update_url: 'http://opendata.iprpraha.cz/DPP/JR/jrdata.zip'
#    departures:
#    - name: transit
#      route: 5
#      stopid: U49N198

  - platform: template
    sensors:
      alenas_phone_battery:
        friendly_name: "Alena's Phone Battery"
        value_template: >-
          {% if is_state('sensor.alenas_iphone_battery_state', 'Charging') %}
            Charging
          {% elif states('sensor.alenas_iphone_battery_level')|float < 15 %}
            "Very Low"
          {% elif states('sensor.alenas_iphone_battery_level')|float < 25 %}
            Low
          {% elif states('sensor.alenas_iphone_battery_level')|float < 40 %}
            "Needs Charging"
          {% elif states('sensor.alenas_iphone_battery_level')|float < 70 %}
            Good
          {% else %}
            Charged
          {% endif %}

  - platform: template
    sensors:
      seans_phone_battery:
        friendly_name: "Sean's Phone Battery"
        value_template: >-
          {% if is_state('sensor.seans_iphone_x_battery_state', 'Charging') %}
            Charging
          {% elif states('sensor.seans_iphone_x_battery_level')|float < 15 %}
            "Very Low"
          {% elif states('sensor.seans_iphone_x_battery_level')|float < 25 %}
            Low
          {% elif states('sensor.seans_iphone_x_battery_level')|float < 40 %}
            "Needs Charging"
          {% elif states('sensor.seans_iphone_x_battery_level')|float < 70 %}
            Good
          {% else %}
            Charged
          {% endif %}

  - platform: waze_travel_time
    origin: !secret home_address
    destination: !secret work_address
    region: 'EU'
    name: Driving Time to Work

  - platform: google_travel_time
    api_key: !secret google_travel_key
    origin: !secret home_address
    destination: !secret work_address
    name: Tram Bus Time to Work
    options:
      mode: transit

  - platform: systemmonitor
    name: hassbian
    resources:
      - type: disk_use_percent
        arg: /
      - type: disk_free
        arg: /
      - type: memory_use_percent
      - type: load_15m

#  - platform: glances
#    name: xanadu
#    host: 10.10.10.20
#    resources:
#      - 'disk_use_percent'
#      - 'disk_free'
#      - 'memory_use_percent'

  - platform: mqtt
    name: "Power Consumption"
    state_topic: "servicelocation/!secret smappee_service_loc/realtime"
    unit_of_measurement: "W"
    value_template: "{{value_json.totalPower}}"
    icon: mdi:power-plug

  - platform: template
    sensors:
      current_rate_power_cost_per_day:
        friendly_name: "Current Electricity Use"
        unit_of_measurement: 'Kč/day'
        value_template: "{{ ((states('sensor.power_consumption')|float * 0.01011816)|round())*10 }}"

binary_sensor:

  - name: Motion Front of House
    platform: hikvision
    host: !secret front_cam_ip
    username: !secret uname
    password: !secret hass_pass

  - name: Motion Right Side of House
    platform: hikvision
    host: !secret right_cam_ip
    username: !secret uname
    password: !secret hass_pass

  - name: Motion Left Side of House
    platform: hikvision
    host: !secret left_cam_ip
    username: !secret uname
    password: !secret hass_pass

  - name: Motion Back of House
    platform: hikvision
    host: !secret back_cam_ip
    username: !secret uname
    password: !secret hass_pass

  - platform: template
    sensors:
      motion_back:
        friendly_name: "Motion - Back"
        device_class: motion
        value_template: >-
          {{ is_state('camera.back', 'recording') }}

#    icon_template: mdi:run
#    entity_picture_template: mdi:run 

  - platform: template
    sensors:
      motion_front:
        friendly_name: "Motion - Front"
        device_class: motion
        value_template: >-
          {{ is_state('camera.front', 'recording') }}
#    icon_template: mdi:run
#    entity_picture_template: mdi:run 

  - platform: template
    sensors:
      motion_left:
        friendly_name: "Motion - Left"
        device_class: motion
        value_template: >-
          {{ is_state('camera.left', 'recording') }}
#    icon_template: mdi:run
#    entity_picture_template: mdi:run 

  - platform: template
    sensors:
      motion_right:
        friendly_name: "Motion - Right"
        device_class: motion
        value_template: >-
          {{ is_state('camera.right', 'recording') }}
#    icon_template: mdi:run
#    entity_picture_template: mdi:run 

switch:

  - platform: command_line
    switches:
      shuffle:
        friendly_name:  Shuffle
        command_on: "curl -X PUT !secret itunes_ip:8181/shuffle -d mode=on"
        command_off: “curl -X PUT !secret itunes_ip:8181/shuffle -d mode=off”
  - platform: command_line
    switches:
      mix:
        friendly_name: Mix
        command_on: "curl -X PUT !secret itunes_ip:8181/playlists/a-45-star/play"
        command_off: "curl -X PUT !secret itunes_ip:8181/playlists/a-45-star/play"
  - platform: command_line
    switches:
      jazz:
        friendly_name: Jazz
        command_on: "curl -X PUT !secret itunes_ip:8181/playlists/best-jazz/play"
        command_off: "curl -X PUT !secret itunes_ip:8181/playlists/best-jazz/play"
  - platform: command_line
    switches:
      rock:
        friendly_name: Rock
        command_on: "curl -X PUT http://!secret itunes_ip:8181/playlists/best-rock/play"
        command_off: "curl -X PUT http://!secret itunes_ip:8181/playlists/best-rock/play"
  - platform: command_line
    switches:
      pop:
        friendly_name: Pop
        command_on: "curl -X PUT http://!secret itunes_ip:8181/playlists/best-pop/play"
        command_off: "curl -X PUT http://!secret itunes_ip:8181/playlists/best-pop/play"
  - platform: command_line
    switches:
      black:
        friendly_name: Black
        command_on: "curl -X PUT http://!secret itunes_ip:8181/playlists/best-black/play"
        command_off: "curl -X PUT http://!secret itunes_ip:8181/playlists/best-black/play"
  - platform: command_line
    switches:
      classical:
        friendly_name: Classical
        command_on: "curl -X PUT http://!secret itunes_ip:8181/playlists/classical/play"
        command_off: "curl -X PUT http://!secret itunes_ip:8181/playlists/classical/play"
  - platform: command_line
    switches:
      electronica:
        friendly_name: Electronica
        command_on: "curl -X PUT http://!secret itunes_ip:8181/playlists/best-electronica/play"
        command_off: "curl -X PUT http://!secret itunes_ip:8181/playlists/best-electronica/play"
  - platform: command_line
    switches:
      kids:
        friendly_name: Kids
        command_on: "curl -X PUT http://!secret itunes_ip:8181/playlists/kids-english/play"
        command_off: "curl -X PUT http://!secret itunes_ip:8181/playlists/kids-english/play"
#  - platform: template
#    switches:
#      all_speakers:
#        turn_on:
#          service: media_player.turn_on
#          data:
#            entity_id: 
#              -  group.all_speakers
#        turn_off:
#          service: media_player.turn_off
#          data:
#            entity_id: 
#              - group.all_speakers

zone:

  - name: Home
    latitude: !secret home_lat
    longitude: !secret home_long
    radius: 150

  - name: Andel
    latitude: 50.069351
    longitude: 14.405687
    radius: 250

  - name: Work
    latitude: !secret work_lat
    longitude: !secret work_long
    radius: 250

evohome:

  username: !secret my_email
  password: !secret hass_pass
  location_idx: 0
  scan_interval: 360 

media_player:

  - platform: itunes
    host: !secret itunes_ip

ios:

  push:
    categories:
      - name: bike_to_work_day
        identifier: 'bikeday'
        actions:
          - identifier: 'BIKE_DAY'
            title: 'Bike Day'

nest:

  client_id: !secret nest_client_id
  client_secret: !secret nest_client_secret
  sensors:
    monitored_conditions:
      - 'co_status'
      - 'smoke_status'
      - 'battery_health'
      - 'color_status'

camera:

  - platform: local_file
    name: Last Object Detected - Left
    file_path: /home/homeassistant/.homeassistant/securitypics/left_latest.jpg

  - platform: local_file
    name: Last Object Detected - Right
    file_path: /home/homeassistant/.homeassistant/securitypics/right_latest.jpg

  - platform: local_file
    name: Last Object Detected - Front
    file_path: /home/homeassistant/.homeassistant/securitypics/front_latest.jpg

  - platform: local_file
    name: Last Object Detected - Back
    file_path: /home/homeassistant/.homeassistant/securitypics/back_latest.jpg

  - platform: synology
    url: http://!secret synology_ip:5000
#    url: https://!secret my_host:5001
    username: !secret syn_uname
    password: !secret hass_pass
    timeout: 120
#    verify_ssl: False

#  - name: Front Outdoor
#    platform: mjpeg
#    mjpeg_url: http://!secret front_cam_ip/Streaming/Channels/2/preview
#    username: !secret uname
#    password: !secret hass_pass
#
#  - name: Right Side Outdoor
#    platform: mjpeg
#    mjpeg_url: http://!secret right_cam_ip/Streaming/Channels/2/preview
#    username: !secret uname
#    password: !secret hass_pass
#
#  - name: Left Side Outdoor
#    platform: mjpeg
#    mjpeg_url: http://!secret left_cam_ip/Streaming/Channels/2/preview
#    username: !secret uname
#    password: !secret hass_pass
#
#  - name: Back Outdoor
#    platform: mjpeg
#    mjpeg_url: http://!secret back_cam_ip/Streaming/Channels/2/preview
#    username: !secret uname
#    password: !secret hass_pass

#  - platform: generic
#    name: WeatherGraph
#    still_image_url: https://www.yr.no/place/Czech_Republic/Prague/Prague/meteogram.svg
#    content_type: 'image/svg+xml'

google:

  client_id: !secret google_email_client_id
  client_secret: !secret google_email_secret

history_graph:
  gr1:
    name: Entrance Temperatures
    entities:
      - climate.entrance
    hours_to_show: 23
    refresh: 60
  gr2:
    name: Dining Room Temperatures
    entities:
      - climate.dining_room
    hours_to_show: 23
    refresh: 60
  gr3:
    name: Living Room Temperatures
    entities:
      - climate.living_room
    hours_to_show: 23
    refresh: 60
  gr4:
    name: Master Bedroom Temperatures
    entities:
      - climate.master_bedroom
    hours_to_show: 23
    refresh: 60
  gr5:
    name: Master Bath Temperatures
    entities:
      - climate.master_bath
    hours_to_show: 23
    refresh: 60
  gr6:
    name: Master Closet Temperatures
    entities:
      - climate.master_closet
    hours_to_show: 23
    refresh: 60
  gr7:
    name: Office & Library Temperatures
    entities:
      - climate.office__library
    hours_to_show: 23
    refresh: 60
  gr8:
    name: Sebastians Room Temperatures
    entities:
      - climate.sebastians_room
    hours_to_show: 23
    refresh: 60
  gr9:
    name: Sophies Room Temperatures
    entities:
      - climate.sophies_room
    hours_to_show: 23
    refresh: 60
  gr10:
    name: Guest Bedroom Temperatures
    entities:
      - climate.guest_bedroom
    hours_to_show: 23
    refresh: 60
  gr11:
    name: Garage Temperatures
    entities:
      - climate.garage
    hours_to_show: 23
    refresh: 60
  gr12:
    name: Basement Temperatures
    entities:
      - climate.basement
    hours_to_show: 23
    refresh: 60
#  gr13:
#    name: Motion Detection Times
#    entities:
#      - binary_sensor.motion_back_of_house_motion
#      - binary_sensor.motion_front_of_house_motion
#      - binary_sensor.motion_left_side_of_house_motion
#      - binary_sensor.motion_right_side_of_house_motion
#    hours_to_show: 12
#    refresh: 60

weather:

  - platform: darksky
    api_key: !secret darksky_key
    name: Weather Forecast
    mode: daily
    scan_interval: 180

  - platform: openweathermap
    api_key: !secret openweathermap_key
    mode: hourly

#anel_iframe:
#cloud9:
#  title: cloud9
#  icon: mdi:wrench
#  url: http://hassbian.local:8181/ide.html

image_processing:

 - platform: tensorflow
   scan_interval: 5
   source:
     - entity_id: camera.back
     - entity_id: camera.right
     - entity_id: camera.left
     - entity_id: camera.front
   model:
     graph: /home/homeassistant/c9workspace/homeassistant/tensorflow/ssd_mobilenet_v2_coco_2018_03_29/frozen_inference_graph.pb
   file_out:
     - "/home/homeassistant/.homeassistant/securitypics/{{ camera_entity.split('.')[1] }}_latest.jpg"
     - "/home/homeassistant/.homeassistant/securitypics/{{ camera_entity.split('.')[1] }}_{{ now().strftime('%Y%m%d_%H%M%S') }}.jpg"   

mqtt:

  broker: !secret pi_ip
  client_id: homeassistant-001
  username: !secret uname
  password: !secret hass_pass

Traceback (if applicable):

If you can tell me where to get it from, I will put it here.. I don't see a core file that looks relevant.. Here are a couple logs, a process listing from when it crashed, and a list of all files named core

home/homeassistant/opencv/build/modules/core
/home/homeassistant/opencv/build/samples/cpp/CMakeFiles/tutorial_mat_the_basic_image_container.dir/tutorial_code/core
/home/homeassistant/opencv/build/samples/cpp/CMakeFiles/tutorial_how_to_scan_images.dir/tutorial_code/core
/home/homeassistant/opencv/build/samples/cpp/CMakeFiles/tutorial_discrete_fourier_transform.dir/tutorial_code/core
/home/homeassistant/opencv/build/samples/cpp/CMakeFiles/tutorial_Drawing_2.dir/tutorial_code/core
/home/homeassistant/opencv/build/samples/cpp/CMakeFiles/tutorial_Drawing_1.dir/tutorial_code/core
/home/homeassistant/opencv/build/samples/cpp/CMakeFiles/tutorial_file_input_output.dir/tutorial_code/core
/home/homeassistant/opencv/build/samples/cpp/CMakeFiles/tutorial_mat_mask_operations.dir/tutorial_code/core
/home/homeassistant/opencv/build/samples/cpp/CMakeFiles/tutorial_interoperability_with_OpenCV_1.dir/tutorial_code/core
/home/homeassistant/opencv/modules/core
/home/homeassistant/opencv/modules/core/include/opencv2/core
/home/homeassistant/opencv/doc/tutorials/core
/home/homeassistant/opencv/samples/cpp/tutorial_code/core
/lib/modules/4.14.90+/kernel/drivers/video/fbdev/core
/lib/modules/4.14.90+/kernel/drivers/gpu/drm/tinydrm/core
/lib/modules/4.14.90+/kernel/sound/core
/lib/modules/4.14.90+/kernel/net/core
/lib/modules/4.14.79-v7+/kernel/drivers/video/fbdev/core
/lib/modules/4.14.79-v7+/kernel/drivers/gpu/drm/tinydrm/core
/lib/modules/4.14.79-v7+/kernel/sound/core
/lib/modules/4.14.79-v7+/kernel/net/core
/lib/modules/4.14.90-v7+/kernel/drivers/video/fbdev/core
/lib/modules/4.14.90-v7+/kernel/drivers/gpu/drm/tinydrm/core
/lib/modules/4.14.90-v7+/kernel/sound/core
/lib/modules/4.14.90-v7+/kernel/net/core
/lib/modules/4.14.79+/kernel/drivers/video/fbdev/core
/lib/modules/4.14.79+/kernel/drivers/gpu/drm/tinydrm/core
/lib/modules/4.14.79+/kernel/sound/core
/lib/modules/4.14.79+/kernel/net/core
/sys/kernel/debug/vchiq/log/core
/proc/sys/net/core
/usr/local/lib/python3.5/dist-packages/numpy/core
/srv/homeassistant/lib/python3.5/site-packages/numpy/core
/srv/homeassistant/lib/python3.5/site-packages/tensorflow/core
/srv/homeassistant/lib/python3.5/site-packages/tensorflow/core/lib/core
/srv/homeassistant/lib/python3.5/site-packages/tensorflow/contrib/autograph/core
/srv/homeassistant/lib/python3.5/site-packages/tensorflow/include/external/aws/aws-cpp-sdk-core/include/aws/core
/srv/homeassistant/lib/python3.5/site-packages/tensorflow/include/tensorflow/core
/srv/homeassistant/lib/python3.5/site-packages/tensorflow/include/tensorflow/core/lib/core
/srv/homeassistant/lib/python3.5/site-packages/tensorboard/plugins/core
/opt/vc/include/interface/mmal/core

root         1  0.0  0.4  26956  4212 ?        Ss   Jan03   0:04 /sbin/init
root         2  0.0  0.0      0     0 ?        S    Jan03   0:00 [kthreadd]
root         4  0.0  0.0      0     0 ?        I<   Jan03   0:00 [kworker/0:0H]
root         6  0.0  0.0      0     0 ?        I<   Jan03   0:00 [mm_percpu_wq]
root         7  0.0  0.0      0     0 ?        S    Jan03   1:22 [ksoftirqd/0]
root         8  0.0  0.0      0     0 ?        I    Jan03   2:24 [rcu_sched]
root         9  0.0  0.0      0     0 ?        I    Jan03   0:00 [rcu_bh]
root        10  0.0  0.0      0     0 ?        S    Jan03   0:00 [migration/0]
root        11  0.0  0.0      0     0 ?        S    Jan03   0:00 [cpuhp/0]
root        12  0.0  0.0      0     0 ?        S    Jan03   0:00 [cpuhp/1]
root        13  0.0  0.0      0     0 ?        S    Jan03   0:00 [migration/1]
root        14  0.0  0.0      0     0 ?        S    Jan03   0:19 [ksoftirqd/1]
root        16  0.0  0.0      0     0 ?        I<   Jan03   0:00 [kworker/1:0H]
root        17  0.0  0.0      0     0 ?        S    Jan03   0:00 [cpuhp/2]
root        18  0.0  0.0      0     0 ?        S    Jan03   0:00 [migration/2]
root        19  0.0  0.0      0     0 ?        S    Jan03   0:18 [ksoftirqd/2]
root        21  0.0  0.0      0     0 ?        I<   Jan03   0:00 [kworker/2:0H]
root        22  0.0  0.0      0     0 ?        S    Jan03   0:00 [cpuhp/3]
root        23  0.0  0.0      0     0 ?        S    Jan03   0:00 [migration/3]
root        24  0.0  0.0      0     0 ?        S    Jan03   0:18 [ksoftirqd/3]
root        26  0.0  0.0      0     0 ?        I<   Jan03   0:00 [kworker/3:0H]
root        27  0.0  0.0      0     0 ?        S    Jan03   0:00 [kdevtmpfs]
root        28  0.0  0.0      0     0 ?        I<   Jan03   0:00 [netns]
root        33  0.0  0.0      0     0 ?        S    Jan03   0:00 [khungtaskd]
root        34  0.0  0.0      0     0 ?        S    Jan03   0:00 [oom_reaper]
root        35  0.0  0.0      0     0 ?        I<   Jan03   0:00 [writeback]
root        36  0.0  0.0      0     0 ?        S    Jan03   0:00 [kcompactd0]
root        37  0.0  0.0      0     0 ?        I<   Jan03   0:00 [crypto]
root        38  0.0  0.0      0     0 ?        I<   Jan03   0:00 [kblockd]
root        39  0.0  0.0      0     0 ?        I<   Jan03   0:00 [watchdogd]
root        40  0.0  0.0      0     0 ?        I<   Jan03   0:00 [rpciod]
root        41  0.0  0.0      0     0 ?        I<   Jan03   0:00 [xprtiod]
root        42  0.0  0.0      0     0 ?        I    Jan03   0:04 [kworker/u8:1]
root        44  0.0  0.0      0     0 ?        S    Jan03   0:31 [kswapd0]
root        45  0.0  0.0      0     0 ?        I<   Jan03   0:00 [nfsiod]
root        55  0.0  0.0      0     0 ?        I<   Jan03   0:00 [kthrotld]
root        56  0.0  0.0      0     0 ?        I<   Jan03   0:00 [iscsi_eh]
root        57  0.0  0.0      0     0 ?        I<   Jan03   0:00 [dwc_otg]
root        58  0.0  0.0      0     0 ?        I<   Jan03   0:00 [DWC Notificatio]
root        59  0.0  0.0      0     0 ?        S<   Jan03   0:00 [vchiq-slot/0]
root        60  0.0  0.0      0     0 ?        S<   Jan03   0:00 [vchiq-recy/0]
root        61  0.0  0.0      0     0 ?        S<   Jan03   0:00 [vchiq-sync/0]
root        62  0.0  0.0      0     0 ?        S    Jan03   0:00 [vchiq-keep/0]
root        63  0.0  0.0      0     0 ?        S<   Jan03   0:00 [SMIO]
root        65  0.0  0.0      0     0 ?        S    Jan03   0:00 [irq/92-mmc1]
root        68  0.3  0.0      0     0 ?        S    Jan03   8:16 [mmcqd/0]
root        69  0.0  0.0      0     0 ?        S    Jan03   1:44 [jbd2/mmcblk0p2-]
root        70  0.0  0.0      0     0 ?        I<   Jan03   0:00 [ext4-rsv-conver]
root        71  0.0  0.0      0     0 ?        I<   Jan03   0:00 [ipv6_addrconf]
root        89  0.0  0.8  35116  7624 ?        Ss   Jan03   2:26 /lib/systemd/systemd-journald
root       125  0.0  0.0  14256    56 ?        Ss   Jan03   0:00 /lib/systemd/systemd-udevd
root       149  0.0  0.0      0     0 ?        I    Jan03   0:00 [kworker/u8:2]
root       151  0.0  0.0      0     0 ?        S    Jan03   0:00 [irq/169-usb-001]
root       210  0.0  0.0      0     0 ?        I<   Jan03   0:00 [cfg80211]
root       213  0.0  0.0      0     0 ?        I<   Jan03   0:00 [brcmf_wq/mmc1:0]
root       214  0.0  0.0      0     0 ?        S    Jan03   0:00 [brcmf_wdog/mmc1]
systemd+   272  0.0  0.0  17280    56 ?        Ssl  Jan03   0:00 /lib/systemd/systemd-timesyncd
nobody     307  0.0  0.0   5296    16 ?        Ss   Jan03   0:01 /usr/sbin/thd --triggers /etc/triggerhappy/triggers.d/ --socket /run/thd.socket --user nobody --deviceglob /dev/input/event*
root       309  0.0  0.1  23748  1364 ?        Ssl  Jan03   0:28 /usr/sbin/rsyslogd -n
root       315  0.0  0.0   5296    60 ?        Ss   Jan03   0:00 /usr/sbin/cron -f
root       326  0.0  0.2   7412  2644 ?        Ss   Jan03   0:00 /lib/systemd/systemd-logind
avahi      329  0.0  0.0   6800   504 ?        Ss   Jan03   1:29 avahi-daemon: running [hassbian.local]
message+   335  0.0  0.0   6508   756 ?        Ss   Jan03   0:00 /usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation
avahi      355  0.0  0.0   6404    24 ?        S    Jan03   0:00 avahi-daemon: chroot helper
root       364  0.0  0.0  27600    96 ?        SLsl Jan03   0:15 /usr/sbin/rngd -r /dev/hwrng
mosquit+   392  0.1  0.0   6620   848 ?        S    Jan03   3:32 /usr/sbin/mosquitto -c /etc/mosquitto/mosquitto.conf
root       416  0.0  0.0  10036    64 ?        Ss   Jan03   0:02 wpa_supplicant -B -c/etc/wpa_supplicant/wpa_supplicant.conf -iwlan0 -Dnl80211,wext
root       432  0.0  0.0      0     0 ?        I<   Jan03   0:00 [kworker/0:1H]
root       433  0.0  0.0      0     0 ?        I<   Jan03   0:00 [kworker/1:1H]
root       443  0.0  0.0      0     0 ?        I<   Jan03   0:00 [kworker/u9:0]
root       445  0.0  0.0      0     0 ?        I<   Jan03   0:00 [kworker/u9:2]
root       446  0.0  0.0   2096     4 ?        S    Jan03   0:00 /usr/bin/hciattach /dev/serial1 bcm43xx 3000000 flow - b8:27:eb:69:d6:19
root       451  0.0  0.0   7272     0 ?        Ss   Jan03   0:00 /usr/lib/bluetooth/bluetoothd
root       507  0.0  0.0   2948    48 ?        Ss   Jan03   0:07 /sbin/dhcpcd -q -w
root       524  0.0  0.0   4196     0 tty1     Ss+  Jan03   0:00 /sbin/agetty --noclear tty1 linux
root       527  0.0  0.0  10212   232 ?        Ss   Jan03   0:00 /usr/sbin/sshd -D
root       545  0.0  0.2  26480  2484 ?        Ss   Jan03   0:09 /usr/sbin/nmbd
root       548  0.0  0.2  40668  2368 ?        Ss   Jan03   0:02 /usr/sbin/smbd
root       552  0.1  0.1  38652   972 ?        S    Jan03   4:46 /usr/sbin/smbd
root       553  0.0  0.0  38664   836 ?        S    Jan03   0:00 /usr/sbin/smbd
root       580  0.0  0.1  40656  1756 ?        S    Jan03   0:00 /usr/sbin/smbd
root       615  0.0  0.0      0     0 ?        I<   Jan03   0:00 [kworker/3:1H]
root       627  0.0  0.0      0     0 ?        I<   Jan03   0:00 [kworker/2:1H]
root     18137  0.0  0.0      0     0 ?        I    15:31   0:00 [kworker/3:1]
root     20731  0.0  0.0      0     0 ?        I    15:35   0:00 [kworker/1:2]
root     20737  0.0  0.0      0     0 ?        I    15:35   0:00 [kworker/2:3]
root     24988  0.0  0.0      0     0 ?        I    15:43   0:00 [kworker/1:0]
root     25157  0.0  0.0      0     0 ?        I    15:54   0:00 [kworker/0:2]
root     25161  0.0  0.0      0     0 ?        I    15:57   0:00 [kworker/2:0]
root     25164  0.0  0.0      0     0 ?        I    16:01   0:00 [kworker/0:0]
root     25167  1.4  1.5  41652 14392 ?        S    16:04   0:04 /usr/sbin/smbd
root     25172  0.0  0.0      0     0 ?        I    16:05   0:00 [kworker/2:1]
root     25174  0.1  0.6  11532  5780 ?        Ss   16:06   0:00 sshd: homeassistant [priv]
homeass+ 25179  0.0  0.5   9656  4992 ?        Ss   16:06   0:00 /lib/systemd/systemd --user
homeass+ 25182  0.0  0.1  28620  1024 ?        S    16:06   0:00 (sd-pam)
homeass+ 25236  0.0  0.3  11532  3648 ?        R    16:06   0:00 sshd: homeassistant@pts/0
homeass+ 25239  0.6  0.4   6236  4264 pts/0    Ss   16:06   0:01 -bash
root     25264  0.0  0.0      0     0 ?        I    16:06   0:00 [kworker/0:1]
root     25270  0.0  0.0      0     0 ?        I    16:06   0:00 [kworker/u8:0]
homeass+ 25289  0.0  0.3   7740  2904 pts/0    R+   16:09   0:00 ps auxww
root     26659  0.0  0.0      0     0 ?        I    14:18   0:00 [kworker/3:0]
[home-assistant-log-freeze.txt](https://github.com/home-assistant/home-assistant/files/2729932/home-assistant-log-freeze.txt)
[crash2.log](https://github.com/home-assistant/home-assistant/files/2729933/crash2.log)

Additional information:

amelchio commented 5 years ago

Since you are on a Pi, try disabling image_processing: and see if that helps with performance.

scstraus commented 5 years ago

I think it may be MQTT as I have a very verbose Smappee Energy meter that likes to update me every second with a very lengthy update which carries very little useful information. I've disabled it for now. I will see if things improve, but already history graphs are working better.

Is there some good way to tell when things are getting overloaded? My CPU and memory usage don't look that bad. Maybe 50% CPU and 60% memory usage, but I suspect it might be I/O where the bottleneck is since I use a lot of graphs.

sebba12 commented 5 years ago

Hey I have a very smiliar problem see #19752, and I do not use MQTT (what I am aware off) I checked my PI's performance and I can't see any problems with it, my avg load is araound 40-50% and spikes only reach 70%. I reverted to version 0.83.X with the same config I used in 0.84 and with version 0.83 I do not have the errors or the craches anymore.

scstraus commented 5 years ago

I also was showing no load issues in terms of CPU or RAM, but I think the database I/O was overwhelmed (Pi's have a slow USB 2.0 bus which is pretty easy to overwhelm). You may want to do a search for "sql" in your log and see if it's complaining about a locked database or something like that. That indicates the database giving up hope ;-).

It doesn't have to be MQTT that overwhelms the database. If you define enough of any kind of sensor, or a few that update a lot you can get this problem, as each update needs a database write to build out the history (and I can't see any way how to reduce the amount of history records being written, but I'd love to know if someone knows how to do that).

sebba12 commented 5 years ago

I have now check for all the logs I can fins both HA-logs and Host logs and I can't find anything with SQL-errors in the loggs

sebba12 commented 5 years ago

Have you been in any luck finding the problem, I am thinking to move the database out from the pi and run it on another server just for testing, do you think the will do any diffrance?

TimothyLeeAdams commented 5 years ago

I suspect it's the calendar platform causing the problem.

I'm having the same issue as of version 0.84.6 as well. I'm running Home Assistant in Docker, so I was able to easily create a test container and was able to isolate components one by one. What I found was that in my setup, calendar components were causing Home Assistant to hang on restart. If I left it long enough, it would eventually start, but that could take upwards of an hour to happen. Once it loaded, the log would be filled entirely with errors about the components taking too long to load. For example: Updating todoist calendar took longer than the scheduled update interval 0:01:00

As a test, created a stock instance of Home Assistant with only the Todoist calendar component loaded and found the same problem, though Home Assistant would now boot after only a few minutes of hanging, again with log errors about taking too long to load. I tried the same test with Google calendar component and had the same problem. As soon as I remove the calendars from my config, Home Assistant seems to boot and run normally again.

@scstraus Can you try removing calendars form your config and let us know what your results are?

scstraus commented 5 years ago

Alright, I will turn them off on the next restart. Since I turned off MQTT, the creashes have stopped, but I still get lots of the "taking over 10 seconds" type messages. I'm just commenting out this. For the record my restarts are only taking about 4 minutes.. I have suspected the calendar component of problems in the past, I've noticed previously that my cameras wouldn't start when calendar was enabled. Sometimes they still don't start, but if I restart enough times it usually works.

google:

client_id: !secret google_email_client_id client_secret: !secret google_email_secret

scstraus commented 5 years ago

So I tried it and I don't see any major change. Still have the "taking over 10 seconds" messages.. I suspect the ones that are left are because my synology is overloaded more than anything else.. I think for me the issue was MQTT with smappee due to it's overzealous refresh rate and large messages.

TimothyLeeAdams commented 5 years ago

@scstraus Okay, thanks for letting me know your results. Seems like there might be a number of components are compounding the issue, though I'm having particular trouble with the calendar component. I'll let you know if I figure anything else out that might be helpful to you.

sebba12 commented 5 years ago

Interesting I tried to disable all my sensors and it makes no different I don't get the update thingy in the logs but the crashes stills happens. Strange thing is that my config now only consist of IKEA, Telldus and Chromecast devices and a vacuum. Anyone can give me a hint on what to try next?

home-assistant / core

Daily crashes preceeded by lots of "Update of xxx is taking over xxx" for lots of components in 0.84.6 #19806