[ZHA] Coordinator setup failing in HA 2024.08.0

[MidnessX]

The problem

Since the update to Home Assistant 2024.08.0 the ZHA integration fails to start due to an exception in the coordinator setup. See the attached logs for details.

What version of Home Assistant Core has the issue?

core-2024.8.0

What was the last working version of Home Assistant Core?

core-2024.7.4

What type of installation are you running?

Home Assistant OS

Integration causing the issue

Zigbee Home Automation

Link to integration documentation on our website

https://www.home-assistant.io/integrations/zha/

Diagnostics information

No response

Example YAML snippet

zha:
  zigpy_config:
    network:
      channel: 11
      channels: [11]
      key: !secret zigbee_network_key
      pan_id: !secret zigbee_pan_id
      extended_pan_id: !secret zigbee_extended_pan_id
    ota:
      ikea_provider: true
      ledvance_provider: true

Anything in the logs that might be useful for us?

2024-08-09 14:26:27.614 ERROR (MainThread) [homeassistant.config_entries] Error setting up entry BV 2010/10 - BV 2010/10 - /dev/ttyUSB0, s/n: [REDACTED] - Silicon Labs - 10C4:8B34 for zha
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 604, in async_setup
    result = await component.async_setup_entry(hass, self)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/zha/__init__.py", line 118, in async_setup_entry
    zha_lib_data: ZHAData = create_zha_config(hass, ha_zha_data)
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/zha/helpers.py", line 1249, in create_zha_config
    flow_control=app_config[CONF_DEVICE][CONF_FLOW_CONTROL],
                 ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
KeyError: 'flow_control'

Additional information

No response

[home-assistant[bot]]

Hey there @dmulcahey, @adminiuga, @puddly, @thejulianjes, mind taking a look at this issue as it has been labeled with an integration (zha) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of `zha` can trigger bot actions by commenting: - `@home-assistant close` Closes the issue. - `@home-assistant rename Awesome new title` Renames the issue. - `@home-assistant reopen` Reopen the issue. - `@home-assistant unassign zha` Removes the current integration label and assignees on the issue, add the integration domain after the command. - `@home-assistant add-label needs-more-information` Add a label (needs-more-information, problem in dependency, problem in custom component) to the issue. - `@home-assistant remove-label needs-more-information` Remove a label (needs-more-information, problem in dependency, problem in custom component) on the issue.

_{^{(message by CodeOwnersMention)}}

---

zha documentation zha source _{^{(message by IssueLinks)}}

[dmulcahey]

can you enable debug logging and attach the home assistant log after ZHA fails to set up please?

[dmulcahey]

Can you also attach the integration diagnostics (settings -> devices and services -> ZHA -> 3 dots menu -> download diagnostics). I think this may potentially be caused by a failed config entry migration a while back.

EDIT for other devs: I am curious to see what this section looks like in the diagnostics:

[TheJulianJES]

If you're unable to download integration diagnostics (as the integration fails to set up), it would be very helpful if you could search for "domain": "zha" in the core.config_entries file. It's in the (hidden) .storage folder in /config. So, the full path should be: /config/.storage/core.config_entries.

There, you'll find a section about the ZHA, please post it entirely. We're mostly interested in the version and data sections.

[MidnessX]

can you enable debug logging and attach the home assistant log after ZHA fails to set up please?

The error message I posted is the only message that gets logged by ZHA in debug mode.

Not sure if you meant setting the log level to debug even for homeassistant.core, but from what I can see it's just a deluge of messages unrelated to ZHA.

Can you also attach the integration diagnostics (settings -> devices and services -> ZHA -> 3 dots menu -> download diagnostics). I think this may potentially be caused by a failed config entry migration a while back.

The download diagnostics option is unavailable from the GUI, so I took a look at the core.config_entries file as requested:

...
"data": {
    "entries": [
      {
        "created_at": "1970-01-01T00:00:00+00:00",
        "data": {
          "device": {
            "path": "/dev/serial/by-id/usb-Silicon_Labs_BV_2010_10_[REDACTED_SN]-if00-port0",
            "baudrate": 115200
          },
          "radio_type": "ezsp"
        },
        "disabled_by": null,
        "domain": "zha",
        "entry_id": "b7a3c15ea5f11f0a5c07231adc92ccf9",
        "minor_version": 1,
        "modified_at": "1970-01-01T00:00:00+00:00",
        "options": {
          "custom_configuration": {
            "zha_options": {
              "default_light_transition": 5.0,
              "enhanced_light_transition": false,
              "light_transitioning_flag": true,
              "always_prefer_xy_color_mode": true,
              "group_members_assume_state": true,
              "enable_identify_on_join": true,
              "consider_unavailable_mains": 7200,
              "consider_unavailable_battery": 21600
            }
          }
        },
        "pref_disable_new_entities": false,
        "pref_disable_polling": false,
        "source": "usb",
        "title": "BV 2010/10 - BV 2010/10 - /dev/ttyUSB0, s/n: [REDACTED_SN] - Silicon Labs - 10C4:8B34",
        "unique_id": "10C4:8B34_[REDACTED_SN]_Silicon Labs_BV 2010/10 - BV 2010/10",
        "version": 4
      },
...

[dmulcahey]

can you reconfigure the coordinator in place and let me know if that gets you running? settings -> devices and services -> zha -> configure -> migrate radio:

select: reconfigure existing radio run through the dialog prompts and when asked about backups select keep existing radio settings.

When this completes hopefully you are up and running. We'll get something done to prevent this in the future.

[eardkdw]

After having (laboriously) restoring HAOS (VirtualBox VM on Ubuntu on a NUC) everything was fine overnight but when I upgraded to 2024.08 this morning my Zigbee network kept re-initialising and eventually stopped working - I had to roll back to 2024.7.3. I'm using the EZSP driver for a (hacked) Lidl Zigbee Gateway, which has been absolutely rock solid up until now. I assumed it was a side-effect of the restore, but my symptoms were very similar to @MidnessX above - it would not reconnect at all, and I got a create_zha_config(hass, ha_zha_data) error - unfortunately I didn't save it....

[MidnessX]

can you reconfigure the coordinator in place and let me know if that gets you running? settings -> devices and services -> zha -> configure -> migrate radio:

select: reconfigure existing radio run through the dialog prompts and when asked about backups select keep existing radio settings.

When this completes hopefully you are up and running. We'll get something done to prevent this in the future.

Yep, that does the trick! Everything is working fine right now.

Thank you all for the prompt support! Truly appreciated.

[eardkdw]

Ah, okay, I tried that and it didn't work. Must be a different issue. My device section, just in case:

"config_entry": {
      "data": {
        "device": {
          "baudrate": 57600,
          "flow_control": "software",
          "path": "socket://192.168.x.x:8888"
        },
        "radio_type": "ezsp"
      },
      "disabled_by": null,
      "domain": "zha",
      "entry_id": "7aa2334e0ca54051646be089b1e6c679",
      "minor_version": 1,
      "options": {},
      "pref_disable_new_entities": false,
      "pref_disable_polling": false,
      "source": "user",
      "title": "socket://192.168.x.x:8888",
      "unique_id": null,
      "version": 4
    },

(only not with x.x obviously :) )

[puddly]

@eardkdw Could you open a separate issue once you grab a debug log of the startup failure? Thanks!

[eardkdw]

Will do, although I'm probably going to await the next update first as it was a pain to sort.

[dmulcahey]

can you reconfigure the coordinator in place and let me know if that gets you running? settings -> devices and services -> zha -> configure -> migrate radio: select: reconfigure existing radio run through the dialog prompts and when asked about backups select keep existing radio settings. When this completes hopefully you are up and running. We'll get something done to prevent this in the future.

Yep, that does the trick! Everything is working fine right now.

Thank you all for the prompt support! Truly appreciated.

Sorry for the issue in the first place. We’ll get something in place to prevent this moving forward.

[puddly]

@MidnessX Did you ever perform a ZHA radio migration to/from this adapter? Do you also remember if you set it up prior to December 2023?

We're trying to figure out how exactly your settings are missing this key.

[MidnessX]

I bought this adapter in 2021. Prior to this I was using a CC2531, but I can't recall if i did a radio migration from it as I'm quite sure I never set that up for use with ZHA (I was using Z2MQTT earlier).

What I am sure about is that I upgraded the BV2010/10 firmware last year, moving from EmberZNet 5.8.1 to EmberZNet 6.7.10. I did not perform a radio migration after the firmware upgrade, but I had to manually change the baud rate in the core.config_entries file.

[akoivist]

Here it was KeyError: 'baudrate', and got fixed by the mentioned procedure: settings -> devices and services -> zha -> configure -> migrate radio: select: reconfigure existing radio run through the dialog prompts and when asked about backups select keep existing radio settings.

[puddly]

@akoivist Can you share your ZHA diagnostics JSON? baudrate definitely shouldn't be missing. We're trying to track down this problem so that nobody has to re-configure their radio to fix it.

[akoivist]

@akoivist Can you share your ZHA diagnostics JSON? baudrate definitely shouldn't be missing. We're trying to track down this problem so that nobody has to re-configure their radio to fix it.

Current working diagnostics

Current working core.config_entries:

{
        "created_at": "1970-01-01T00:00:00+00:00",
        "data": {
          "device": {
            "path": "/dev/serial/by-id/usb-Nabu_Casa_SkyConnect_v1.0_b0fcf35fd193ed11b0a27ef23b20a988-if00-port0",
            "baudrate": 115200,
            "flow_control": "software"
          },
          "radio_type": "ezsp"
        },
        "disabled_by": null,
        "domain": "zha",
        "entry_id": "e34883fe4e5e2217b74276e56c46a73c",
        "minor_version": 1,
        "modified_at": "2024-08-10T06:59:22.488908+00:00",
        "options": {},
        "pref_disable_new_entities": false,
        "pref_disable_polling": false,
        "source": "user",
        "title": "skyconnect",
        "unique_id": null,
        "version": 4
      },

Previous (2024.7.x) working core.config_entries:

{
        "data": {
          "device": {
            "path": "/dev/serial/by-id/usb-Nabu_Casa_SkyConnect_v1.0_b0fcf35fd193ed11b0a27ef23b20a988-if00-port0"
          },
          "radio_type": "ezsp"
        },
        "disabled_by": null,
        "domain": "zha",
        "entry_id": "e34883fe4e5e2217b74276e56c46a73c",
        "minor_version": 1,
        "options": {
          "custom_configuration": {
            "zha_options": {
              "group_members_assume_state": false,
              "enhanced_light_transition": true,
              "default_light_transition": 0,
              "light_transitioning_flag": true,
              "always_prefer_xy_color_mode": true,
              "enable_identify_on_join": true,
              "consider_unavailable_mains": 7200,
              "consider_unavailable_battery": 21600
            }
          }
        },
        "pref_disable_new_entities": false,
        "pref_disable_polling": false,
        "source": "user",
        "title": "skyconnect",
        "unique_id": null,
        "version": 4
      },

[TheJulianJES]

@akoivist Do you remember if you used Multiprotocol in the past and then migrated to ZHA? Or how/when you set up ZHA in general?

[akoivist]

@akoivist Do you remember if you used Multiprotocol in the past and then migrated to ZHA? Or how/when you set up ZHA in general?

No I have not used Multiprotocol firmware with Skyconnect. I originally setup ZHA and Skyconnect in another machine and connected to it via IP connection from this host. Then moved the stick to this host and changed the connection. Can't recall how I did that though. Maybe editing directly the config file...