Group states are marked as unknown in 0.116.0b1

philhawthorne commented 4 years ago

The problem

When Home Assistant 0.116.0b1 is loaded, group states are marked as unknown instead of their member values.

For example:

a group with just device_trackers will be marked as home or not_home. Now marked as unknown
a group with just cover entities will be marked as open or closed. Now marked as unknown
a group with just light entities would be marked as on or off. Now marked as unknown

Environment

Home Assistant Core release with the issue: 0.116.0b1
Last working Home Assistant Core release (if known): 0.115.x
Operating environment (OS/Container/Supervised/Core): Container
Integration causing this issue: Groups
Link to integration documentation on our website: https://www.home-assistant.io/integrations/group/

Problem-relevant `configuration.yaml`

group:
  blinds_livingroom:
    name: Living Room Blinds
    entities:
      - cover.living_room_right
      - cover.living_room_middle
      - cover.living_room_left

Traceback/Error logs

Additional information

bdraco commented 4 years ago

I tried


master_drapes:
    name: Master Drapes
    entities:
      - cover.master_bedroom_south_drapes
      - cover.master_bedroom_west_drape

and it appears to be working as expected:

Is Home Assistant stuck starting by chance?

towerhand commented 4 years ago

I can confirm, this is happening to me too since b0. Most group are unknown after a restart.

bdraco commented 4 years ago

Are there any errors in the log? The state should be calculated as soon as the start event fires.

bdraco commented 4 years ago

If there aren't any errors it would be helpful to get debug logs for homeassistant.core for affected and unaffected groups so we can determine the difference that is causing the unknown state.

bdraco commented 4 years ago

logger:
  default: warning
  logs:
     homeassistant.core: debug

philhawthorne commented 4 years ago

Nah not stuck loading for me.

With debug set, this is the only log I get for the group below

2020-10-02 12:35:58 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=group.living_room_downlights, old_state=None, new_state=<state group.living_room_downlights=unknown; entity_id=('light.living_front_ri', 'light.living_back_lef', 'light.living_back_cen', 'light.living_front_le', 'light.living_front_ce', 'light.living_back_rig'), order=25, friendly_name=Living Room Downlights @ 2020-10-02T12:35:58.265467+10:00>>

bdraco commented 4 years ago

What are the states of the 6 lights in the group?

philhawthorne commented 4 years ago

Off

However for things like blinds/cover groups that are doing the same thing, their state is on (aka "open")

towerhand commented 4 years ago

For me, the state of the individual entities is correct, only the group states show as unknown. I can try the debug settings for the logs tomorrow morning.

bdraco commented 4 years ago

I wrote tests for the specific light case. https://github.com/home-assistant/core/pull/41034

Unfortunately everything seems to work as expected. It seems like async_process_integration_platforms is having trouble loading group.py for each integration. I wonder if there is a custom integration that has a group.py in it that is causing it to fail?

philhawthorne commented 4 years ago

I'm using

HACS
Alexa Media Player
Remote Home Assistant

I disabled each integration one-by-one, until my HA logs were empty of "You are using a custom integration..." warnings. I even went as far as renaming custom_components folder to something else, to ensure that none would be loaded.

However the group states are still not updating for me. So even running Home Assistant with no custom components, not all groups are updating their states.

I've noticed that groups which contain a Vera device, do update. I am guessing that as Vera device updates come in this triggers the group state to be re-rendered. However groups with entities only from Deconz don't get updated.

Groups with other items, like template sensors, device trackers etc also don't update.

I tried doing a call with homeassistant.update_entity on a group with only Deconz devices, but that didn't update the state of the group.

2020-10-02 13:38:21 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event call_service[L]: domain=homeassistant, service=update_entity, service_data=entity_id=group.exterior_lights>

So for the moment I am stumped. Perhaps there is a common integration @towerhand and I are using which is causing this?

towerhand commented 4 years ago

I mainly noticed it because I have some automations for my garage doors so they were getting triggered due to the unknown state in the group, the state gets fixed if I force or wait the update of one of the entities in the group. For custom components I have hacs, adaptive lighting and smartthinq.

probot-home-assistant[bot] commented 4 years ago

group documentation group source _{^{(message by IssueLinks)}}

probot-home-assistant[bot] commented 4 years ago

Hey there @home-assistant/core, mind taking a look at this issue as its been labeled with an integration (group) you are listed as a codeowner for? Thanks! _{^{(message by CodeOwnersMention)}}

balloob commented 4 years ago

Do you see any other errors in your logs ?

philhawthorne commented 4 years ago

With debug level set to debug on core, these are the only errors I see

2020-10-02 19:52:01 ERROR (MainThread) [homeassistant.components.sensor] Platform mqtt does not generate unique IDs. ID soma_c9760c4577db_battery already exists - ignoring sensor.bedroom_blinds_battery
2020-10-02 19:52:01 ERROR (MainThread) [homeassistant] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/alexa/state_report.py", line 150, in async_send_add_or_update_message
    alexa_entity = ENTITY_ADAPTERS[domain](hass, config, hass.states.get(entity_id))
  File "/usr/src/homeassistant/homeassistant/components/alexa/entities.py", line 220, in __init__
    self.entity_conf = config.entity_config.get(entity.entity_id, {})
AttributeError: 'NoneType' object has no attribute 'entity_id'
2020-10-02 19:52:14 ERROR (MainThread) [homeassistant] Error doing job: Task was destroyed but it is pending!

Doubt they are related

bdraco commented 4 years ago

Do you have any custom_components with a group.py ?

ls -l custom_components/*/group.py ?

philhawthorne commented 4 years ago

Do you have any custom_components with a group.py ?

Nope, none have a group.py. I also did a fresh run of Home Assistant with all custom components disabled, which didn't solve it.

towerhand commented 4 years ago

This is what I have searching for group with the debug logs for core. Also no group.py in the custom_components.

2020-10-02 07:40:57 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event component_loaded[L]: component=config.group>

2020-10-02 07:40:58 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=group.garage_doors, old_state=None, new_state=<state group.garage_doors=unknown; entity_id=('cover.main_garage_door_opener', 'cover.truck_garage_door_opener'), order=1, friendly_name=Garage Doors Group @ 2020-10-02T07:40:58.987537-05:00>>
2020-10-02 07:40:58 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=group.door_sensors, old_state=None, new_state=<state group.door_sensors=unknown; entity_id=('binary_sensor.patio_door_contact', 'binary_sensor.front_door_contact', 'binary_sensor.garage_door_contact'), order=2, friendly_name=Door Sensors Group @ 2020-10-02T07:40:58.987659-05:00>>
2020-10-02 07:40:58 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=group.window_sensors, old_state=None, new_state=<state group.window_sensors=unknown; entity_id=('binary_sensor.left_office_window_contact', 'binary_sensor.right_office_window_contact', 'binary_sensor.left_great_room_window_sensor_contact', 'binary_sensor.right_great_room_window_sensor_contact', 'binary_sensor.kitchen_window_sensor_contact'), order=3, friendly_name=Window Sensors Group @ 2020-10-02T07:40:58.987772-05:00>>
2020-10-02 07:40:58 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=group.motion_sensors, old_state=None, new_state=<state group.motion_sensors=unknown; entity_id=('binary_sensor.foyer_motion_occupancy', 'binary_sensor.mudroom_motion_occupancy', 'binary_sensor.kitchen_motion_occupancy'), order=4, friendly_name=Motion Sensors group @ 2020-10-02T07:40:58.987875-05:00>>
2020-10-02 07:40:58 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=group.leak_sensors, old_state=None, new_state=<state group.leak_sensors=unknown; entity_id=('binary_sensor.kitchen_sink_water_leak', 'binary_sensor.washer_sensor_water_leak'), order=5, friendly_name=Leak Sensors Group @ 2020-10-02T07:40:58.987983-05:00>>
2020-10-02 07:40:58 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=group.alarm_sensors, old_state=None, new_state=<state group.alarm_sensors=unknown; entity_id=('binary_sensor.patio_door_contact', 'binary_sensor.front_door_contact', 'binary_sensor.garage_door_contact', 'binary_sensor.foyer_motion_occupancy', 'binary_sensor.mudroom_motion_occupancy', 'binary_sensor.kitchen_motion_occupancy', 'binary_sensor.left_office_window_contact', 'binary_sensor.right_office_window_contact', 'binary_sensor.left_great_room_window_sensor_contact', 'binary_sensor.right_great_room_window_sensor_contact', 'binary_sensor.kitchen_window_sensor_contact', 'cover.main_garage_door_opener', 'cover.truck_garage_door_opener'), order=6, friendly_name=Alarm group @ 2020-10-02T07:40:58.988095-05:00>>
2020-10-02 07:40:58 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=group.mainfloor_lights, old_state=None, new_state=<state group.mainfloor_lights=unknown; entity_id=('light.kitchen_lights_dimmer', 'switch.kitchen_island_lights_switch', 'switch.kitchen_pendant_light', 'light.kitchen_cabinet_lights', 'switch.dining_room_lights_switch', 'light.great_room_lights_dimmer', 'switch.office_lights_switch', 'light.xiaomi_mi_led_desk_lamp', 'switch.garage_lights_switch', 'switch.mudroom_lights_switch', 'switch.foyer_lights_switch', 'light.great_room_lamps'), order=7, friendly_name=Mainfloor Lights @ 2020-10-02T07:40:58.988221-05:00>>
2020-10-02 07:40:58 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=group.upstairs_lights, old_state=None, new_state=<state group.upstairs_lights=unknown; entity_id=('light.master_bedroom_lights_dimmer', 'light.luis_bedside_lamp', 'switch.laura_bedside_lamp', 'switch.master_bathroom_shower_lights', 'switch.master_bathroom_vanity_lights', 'switch.stairwell_lights', 'switch.hallway_lights'), order=8, friendly_name=Upstairs Lights @ 2020-10-02T07:40:58.988337-05:00>>
2020-10-02 07:40:58 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=group.master_bedroom, old_state=None, new_state=<state group.master_bedroom=unknown; entity_id=('light.master_bedroom_lights_dimmer', 'light.luis_bedside_lamp', 'switch.laura_bedside_lamp'), order=9, friendly_name=Master Bedroom Lights @ 2020-10-02T07:40:58.988443-05:00>>
2020-10-02 07:40:58 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=group.wax_warmers, old_state=None, new_state=<state group.wax_warmers=unknown; entity_id=('switch.wax_warmer_foyer', 'switch.wax_warmer_great_room', 'switch.wax_warmer_mudroom', 'switch.wax_warmer_kitchen', 'switch.wax_warmer_guest_bathroom', 'switch.wax_warmer_upstairs_bathroom'), order=10, friendly_name=Wax Warmers @ 2020-10-02T07:40:58.988544-05:00>>
2020-10-02 07:40:58 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=group.vacuum, old_state=None, new_state=<state group.vacuum=unknown; entity_id=('input_select.vacuum_room_select', 'script.vacuum_room'), order=11, friendly_name=Vacuum a room @ 2020-10-02T07:40:58.988639-05:00>>

2020-10-02 07:40:59 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event service_registered[L]: domain=group, service=reload>
2020-10-02 07:40:59 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event service_registered[L]: domain=group, service=set>
2020-10-02 07:40:59 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event service_registered[L]: domain=group, service=remove>
2020-10-02 07:40:59 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event component_loaded[L]: component=group>

2020-10-02 07:41:05 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=group.mainfloor_lights, old_state=<state group.mainfloor_lights=unknown; entity_id=('light.kitchen_lights_dimmer', 'switch.kitchen_island_lights_switch', 'switch.kitchen_pendant_light', 'light.kitchen_cabinet_lights', 'switch.dining_room_lights_switch', 'light.great_room_lights_dimmer', 'switch.office_lights_switch', 'light.xiaomi_mi_led_desk_lamp', 'switch.garage_lights_switch', 'switch.mudroom_lights_switch', 'switch.foyer_lights_switch', 'light.great_room_lamps'), order=7, friendly_name=Mainfloor Lights @ 2020-10-02T07:40:58.988221-05:00>, new_state=<state group.mainfloor_lights=off; entity_id=('light.kitchen_lights_dimmer', 'switch.kitchen_island_lights_switch', 'switch.kitchen_pendant_light', 'light.kitchen_cabinet_lights', 'switch.dining_room_lights_switch', 'light.great_room_lights_dimmer', 'switch.office_lights_switch', 'light.xiaomi_mi_led_desk_lamp', 'switch.garage_lights_switch', 'switch.mudroom_lights_switch', 'switch.foyer_lights_switch', 'light.great_room_lamps'), order=7, friendly_name=Mainfloor Lights @ 2020-10-02T07:41:05.202955-05:00>>
2020-10-02 07:41:05 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=group.upstairs_lights, old_state=<state group.upstairs_lights=unknown; entity_id=('light.master_bedroom_lights_dimmer', 'light.luis_bedside_lamp', 'switch.laura_bedside_lamp', 'switch.master_bathroom_shower_lights', 'switch.master_bathroom_vanity_lights', 'switch.stairwell_lights', 'switch.hallway_lights'), order=8, friendly_name=Upstairs Lights @ 2020-10-02T07:40:58.988337-05:00>, new_state=<state group.upstairs_lights=on; entity_id=('light.master_bedroom_lights_dimmer', 'light.luis_bedside_lamp', 'switch.laura_bedside_lamp', 'switch.master_bathroom_shower_lights', 'switch.master_bathroom_vanity_lights', 'switch.stairwell_lights', 'switch.hallway_lights'), order=8, friendly_name=Upstairs Lights @ 2020-10-02T07:41:05.647447-05:00>>
2020-10-02 07:41:05 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=group.master_bedroom, old_state=<state group.master_bedroom=unknown; entity_id=('light.master_bedroom_lights_dimmer', 'light.luis_bedside_lamp', 'switch.laura_bedside_lamp'), order=9, friendly_name=Master Bedroom Lights @ 2020-10-02T07:40:58.988443-05:00>, new_state=<state group.master_bedroom=on; entity_id=('light.master_bedroom_lights_dimmer', 'light.luis_bedside_lamp', 'switch.laura_bedside_lamp'), order=9, friendly_name=Master Bedroom Lights @ 2020-10-02T07:41:05.647571-05:00>>
2020-10-02 07:44:45 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=group.window_sensors, old_state=<state group.window_sensors=unknown; entity_id=('binary_sensor.left_office_window_contact', 'binary_sensor.right_office_window_contact', 'binary_sensor.left_great_room_window_sensor_contact', 'binary_sensor.right_great_room_window_sensor_contact', 'binary_sensor.kitchen_window_sensor_contact'), order=3, friendly_name=Window Sensors Group @ 2020-10-02T07:40:58.987772-05:00>, new_state=<state group.window_sensors=off; entity_id=('binary_sensor.left_office_window_contact', 'binary_sensor.right_office_window_contact', 'binary_sensor.left_great_room_window_sensor_contact', 'binary_sensor.right_great_room_window_sensor_contact', 'binary_sensor.kitchen_window_sensor_contact'), order=3, friendly_name=Window Sensors Group @ 2020-10-02T07:44:45.589592-05:00>>
2020-10-02 07:44:45 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=group.alarm_sensors, old_state=<state group.alarm_sensors=unknown; entity_id=('binary_sensor.patio_door_contact', 'binary_sensor.front_door_contact', 'binary_sensor.garage_door_contact', 'binary_sensor.foyer_motion_occupancy', 'binary_sensor.mudroom_motion_occupancy', 'binary_sensor.kitchen_motion_occupancy', 'binary_sensor.left_office_window_contact', 'binary_sensor.right_office_window_contact', 'binary_sensor.left_great_room_window_sensor_contact', 'binary_sensor.right_great_room_window_sensor_contact', 'binary_sensor.kitchen_window_sensor_contact', 'cover.main_garage_door_opener', 'cover.truck_garage_door_opener'), order=6, friendly_name=Alarm group @ 2020-10-02T07:40:58.988095-05:00>, new_state=<state group.alarm_sensors=off; entity_id=('binary_sensor.patio_door_contact', 'binary_sensor.front_door_contact', 'binary_sensor.garage_door_contact', 'binary_sensor.foyer_motion_occupancy', 'binary_sensor.mudroom_motion_occupancy', 'binary_sensor.kitchen_motion_occupancy', 'binary_sensor.left_office_window_contact', 'binary_sensor.right_office_window_contact', 'binary_sensor.left_great_room_window_sensor_contact', 'binary_sensor.right_great_room_window_sensor_contact', 'binary_sensor.kitchen_window_sensor_contact', 'cover.main_garage_door_opener', 'cover.truck_garage_door_opener'), order=6, friendly_name=Alarm group @ 2020-10-02T07:44:45.589820-05:00>>

bdraco commented 4 years ago

I think this is a load order issue.

bdraco commented 4 years ago

Thanks for the logs. I'm pretty sure I've solved it in #41034 as I was able to get the unknown state in the test unexpectedly.

towerhand commented 4 years ago

Just to make sure you are aware, this happens with covers and sensors too.

bdraco commented 4 years ago

Just to make sure you are aware, this happens with covers and sensors too.

Thanks. It could happen for any integration that loaded after group.

rccoleman commented 4 years ago

I was seeing this too after updating and it was fixed for my "single-level" groups (which only include lights/switches directly) when I hit "reload groups" in Server Controls. My "nested" groups which include other groups are still misbehaving in the same way, though (constituent parts are correct, top-level is "unknown").

In short, groups that reference groups are also problematic.

bdraco commented 4 years ago

I was seeing this too after updating and it was fixed for my "single-level" groups (which only include lights/switches directly) when I hit "reload groups" in Server Controls. My "nested" groups which include other groups are still misbehaving in the same way, though (constituent parts are correct, top-level is "unknown").

In short, groups that reference groups are also problematic.

I couldn't get that to break after https://github.com/home-assistant/core/pull/41034. Were you testing the linked PR or 0.116b1 ?

rccoleman commented 4 years ago

Just 0.116.0b1. I'll pull https://github.com/home-assistant/core/pull/41034 in and confirm.

rccoleman commented 4 years ago

Hmm. I'm still seeing the bad behavior from my super-groups after pulling in that commit (copying the whole homeassistant/components/group into /config/custom_components, updating __init__.py, and ensuring that it's being loaded). The other "simple" groups all work fine with the new changes.

This is a simple example from my groups.yaml:

  dr_group:
    name: Dining Room Group
    entities:
      - group.dr

  dr:
    name: Dining Room
    entities:
      - switch.dr_lights_scene

and this is the Lovelace YAML:

                - type: entities
                  name: test
                  entities:
                    - group.dr_group
                    - group.dr

No groups.py in custom_components, either. I'm using:

➜  custom_components ls
__pycache__          audiconnect          group                plex_recently_added  smartweather
aarlo                enable_debug.py      hacs                 pyscript
alexa_media          garbage_collection   lock-manager         simpleicons

bdraco commented 4 years ago

I wrote tests for a group that references another group and some entities that are not groups but didn't for a group of groups. That isn't something I'd really recommend since it leads to chains of state changed events which aren't very efficient. I'll see if I can replicate the group of groups issue tomorrow.

rccoleman commented 4 years ago

Thanks, appreciate it. I use them in "are any lights on?" and "turn off all lights" situations in an attempt to consolidate all lights on a floor by organizing by rooms. It seems like the alternative would be a group per room for local control (already have this) and then replicating the entire entity list in a larger group, both error-prone and requiring additional maintenance. It's also nice to click on the supergroup/floor in Lovelace and see/manipulate the states of the subgroups/rooms, something that would be lost if it just turns into a giant list of entities.

bdraco commented 4 years ago

Its a bit more code than I had hoped, but I fixed the group of a group case in the PR

rccoleman commented 4 years ago

Can confirm that it works now. Thanks!

home-assistant / core