[Feature] Notify OTA updates

[pdecat]

It would be awesome to be able to hook into ZHA to be able to trigger automations when OTA update events occur.

My use case: I've had some issues recently with some IKEA devices behaving weirdly and it took me time to understand they were caused by recent OTA updates.

PS: not sure it is even feasible without modifying ZHA.

[mdeweerd]

There seems to be set of ZHA events that could help reveal such a case - by "set" I mean: all signals ending in _zha_update_device .

Other than that, one would have to listen in on the Ota cluster events, probably in a way similar to ZHA core and generate a signal on the appropriate commands.

https://github.com/home-assistant/core/blob/dev/homeassistant/components/zha/core/channels/general.py#L371-L386

grep: ./core/channels/.general.py.swp: binary file matches
./core/channels/general.py:    SIGNAL_UPDATE_DEVICE,
./core/channels/general.py:            self.async_send_signal(SIGNAL_UPDATE_DEVICE.format(signal_id), args[3])
./core/const.py:SIGNAL_UPDATE_DEVICE = "{}_zha_update_device"
./core/device.py:    SIGNAL_UPDATE_DEVICE,
./core/device.py:                SIGNAL_UPDATE_DEVICE.format(zha_dev.channels.unique_id),

[Hedda]

My use case: I've had some issues recently with some IKEA devices behaving weirdly and it took me time to understand they were caused by recent OTA updates.

+1 as because of issues like those many users will not really want to always leave OTA enabled once have enabled it for a manufacturer. So it would be better if could get an information update in UI somewhere that OTA updates are available for...

https://www.home-assistant.io/integrations/zha#ota-firmware-updates

PS: not sure it is even feasible without modifying ZHA.

IMHO best would be if OTA configuration as well as OTA update availability information for ZHA could be moved to HA's UI, as it would be a user-friendly experience for end-users if they could from GUI choose to enable or disable OTA per manufacturer or device.

[mdeweerd]

super user friendly for end-users if they could from GUI choose to enable or disable OTA per manufacturer or device.

Ideally it would be as for any other update: indicate that one is available and validate the update.
It would also be user friendly that manufacturers ensure backward compatibility. I do not understand why they had to break this. If Ikea (or its subcontractor) has issues with group commands being unreliable, they should have updated their controller/coordinator and not the device itself.

Any way, at the ZHA_TOOLKIT level there is currently the undocumented ota_notify command (from zha_custom) that sets up sw_build_id reporting and sends an "image_notify" to the device to let it know that a new image is available.

I do not intend to add a ui interface to zha_toolkit other than the Developer > Service interface or the configuration of the integration (not existing today). But I am open to integrate code that does so.

Further, notifying/blocking OTA upgrades requires adding a listener into ZHA essentially.

[MattWestb]

IKEA was updating there controllers then they is using touch link for paring device in there system and they need first doing changes in the controller for not braking the system completely then changing the parameters for the network and then roiling out the light devices and in the posses they was braking the group binding Its not so easy saying how and what they shall have doing for getting it right but there ZCB is running on on old 5.7.2 stack and the new standard for updated devices is 6.3.1 (only 3 controllers is not updated to it but all lights is now) for first and second gen devices and for 3 gen they is using 6.10.2.

I think puddly is having some thinking implanting little more advance function then hi have time and using the Zigbee.db and saving the current and "wanted" version if the device firmware so can "locking" firmware for problematic devices and also rolling back for testing without hacking the system code. I hope hi is doing it then all other problem is fixed hi is having on his table for the moment.

I think IKEA is using one little "light" version for queering the device for update and most other is needing more parameters for getting the device responding on the request.

[Hedda]

I guess another problem is if a user does update firmware via OTA and then discover problems so that they want to roll-back to an old version, were then to get older OTA firmware images online if do not have them saved/cached on your local setup?

Koenkk does keep OTA firmware images in https://github.com/Koenkk/zigbee-OTA the brands that do not publically publish OTA images via a server with public access, but he does not keep OTA images for brands that do have servers with public access.

Do they keep any old OTA images online even after they published newer OTA image versions for a device? Those publishing their OTA images on publicly accessible servers today being; IKEA, LEDVANCE/OSRAM, SALUS/Computime, Ubisys, Securifi, and Lixee.

I mean if you want to roll back to an older OTA image version do you need to manually hunt the older OTA firmware down you want to use and then somehow manually force flash an older version (as normally OTA will only automatically update to newer version?)?

[MattWestb]

I have found one very old OTA from IKEA ZLL that is stored in GIT-HUB but i have nearly all from the last 3 years that is being downloaded with my de(F)CONZ install and can serving user if they like One way is saving the json from IKEA then its looks like they is keeping the old images online but its no grantee that they stay there for ever.

Some firmware is blocked for downgrading (i think all IKEA controllers) but some is not (IKEA light was not) but Zigpy need patching then its allays trying the newest one but its described in the wiki.

The puddly way is automatic updating to the latest if the user not have disabling or forcing one "custom version" for the device and if cant finding it online it must being stored local in the system for working OK.

[mdeweerd]

The feature requested is not to "prevent" automatic updates but notify when a new OTA version is available.

I did an OTA update this night and therefore examined the pre-existing "ota_notify" code.

My configuration was already set up to point to a local (empty) "otau" directory, and upon adding the ota file, the update did no happen. In fact the OTA Provider's image lists are updated only when HA starts up, so new files are not detected...

So I added a hack to zha-toolkit to make the OTA Providers update the image lists prior to notifying the device. It can also be called using the execute service and the ota_update_images command.

After that the upgrade of the device worked without ever restarting HA (not even for updating zha-toolkit).

So in light of the feature request, zha-toolkit can now "trigger" the update of the OTA Image lists. That is required to be able to detect that there is a new image. However, as far as I understand it, the OTA provider is either defined - in which case it will also result in automatic updates - or not defined - in which case the images can not be fetched because the provider is not defined.

ZHA Toolkit could probably hack the base zigpy code by substituting the appropriate function to catch the OTA update request from a device. At that time, this request could result in a version check (possibly done by the original code) which would could result in an event generated by ZHA Toolkit, and instead of forwarding the positive answer to the device, replace it with a "negative" answer.

Then when the user uses "ota_notify", the device that is subject of the OTA notification could be added to a dynamic list in ZHA (with a time limit) such that the functionnality described above does not convert the positive answer in a negative one.

Replacing a function is fairly easy to do. I already did it in the toolkit (may not be used at this time): https://github.com/mdeweerd/zha-toolkit/blob/main/custom_components/zha_toolkit/zcl_attr.py#L19-L42

I just set up the following code to demonstrate this:

class BaseClass:
    def __init__(self, name):
        self.name=name
    def OriginalFunctionA(self):
        print(f"Hello from original A: {self.name}")

    def OriginalFunctionB(self):
        print(f"Hello from original B: {self.name}")

# Define our hack functions
def ReplaceHack(self):
    print(f"Hello from ReplaceHack: {self.name}")

orgB=BaseClass.OriginalFunctionB
def WrapHack(self):
    print(f"Entry of WrapHack: {self.name}")
    orgB(self)
    print(f"Exit from WrapHack: {self.name}")

# Now actually hack the original class
BaseClass.OriginalFunctionA=ReplaceHack
BaseClass.OriginalFunctionB=WrapHack

inst1=BaseClass("Inst1")
inst2=BaseClass("Inst2")

inst1.OriginalFunctionA()
inst1.OriginalFunctionB()
inst2.OriginalFunctionA()
inst2.OriginalFunctionB()

The output is:

Hello from ReplaceHack: Inst1
Entry of WrapHack: Inst1
Hello from original B: Inst1
Exit from WrapHack: Inst1
Hello from ReplaceHack: Inst2
Entry of WrapHack: Inst2
Hello from original B: Inst2
Exit from WrapHack: Inst2

So the OriginalFunctionA function is effectively replaced. The OriginalFunctionB function is effectively wrapped. The replacements have access to the class's properties/attributes.

[pdecat]

The feature requested is to "prevent" automatic updates but notify when a new OTA version is available.

Well, I originally meant to be able get notified of automatic updates, pending or occurring, not really preventing them. But what you're proposing could work even better for me given that there aren't that many firmware updates, as I could then decide to trigger them only when I'm confident I have time to troubleshoot potential issues.

So I added a hack to zha-toolkit to make the OTA Providers update the image lists prior to notifying the device. It can also be called using the execute service and the ota_update_images command.

That was the next item on my wish list, thanks!

[mdeweerd]

I mean to write that the original request is 'not to prevent' (I updated my post).

There is no real use IMHO to just be notified about updates because then its too late - you merely know that your device is possibly behaving differently because of an update.

[pdecat]

There is no real use IMHO to just be notified about updates because then its too late - you merely know that your device is possibly behaving differently because of an update.

Maybe too late, but you know it happened, and it would prevent loosing hours wandering forums and issue trackers looking for some hint :)

[Hedda]

Did you guys read/see that the new Home Assistant 2022.4 release introduced a brand new entity type update for this purpose?

https://www.home-assistant.io/blog/2022/04/06/release-20224/#introducing-update-entities

Introducing update entities

Say “hi!” to a brand new entity type: update.

Update entities can tell you if an update is available for your device and service and, in some cases, allow you to install the update straight from Home Assistant!

And the beautiful thing is: They show up in your configuration Dashboard, just like a Home Assistant, Home Assistant OS, or add-on update. As a matter of fact, those are now update entities too!

Now updates are entities, other integrations can provide them too! For example, a WLED firmware update for your LED strip will show up and installs with a single click.

https://www.home-assistant.io/images/blog/2022-04/update-install.gif

Screen recording showing a WLED firmware update can be installed with a clickA WLED firmware update can now be installed just like any other update.

These update entities can be provided by integrations and provide more information about the update, like: Version information, links to release notes, and the possibility to skip that specific version offered.

The blue number indicator on the configuration cog icon in your sidebar tells you how many updates are pending.

The Home Assistant Supervisor (providing Core, OS, and add-on updates), WLED, Pi-hole, and Synology DSM integrations have implemented these brand new update entities in this release.

And because they are all entities, it means you can automate with them. For example, you could send a notification when a new add-on update is available for installation.

[Hedda]

Support for this new update entity type would of course have to be added to the ZHA component inside Home Assistant core:

https://www.home-assistant.io/integrations/update

https://www.home-assistant.io/integrations/#updates

[mdeweerd]

@Hedda Thanks for highlighting this new feature - I agree that this is something that is something that would need to be in ZHA and allowed by zigpy. Currently the zigbee updates are done juste because they're available.

[Hedda]

Posted a question about it to ZHA developers from the zigpy project here -> https://github.com/zigpy/zigpy/discussions/951

[Hedda]

Posted a question about it to ZHA developers from the zigpy project here -> zigpy/zigpy#951

FYI, this is now been implemented into zigpy and Home Assistant core, but note support for different JSON schemas also changed:

https://www.home-assistant.io/blog/2024/02/07/release-20242/#updating-your-zigbee-devices

https://github.com/home-assistant/core/pull/107612

https://github.com/home-assistant/core/pull/111159

https://github.com/zigpy/zigpy/pull/1321

https://github.com/zigpy/zigpy/pull/1340

https://github.com/zigpy/zigpy/pull/1354

zigpy wiki and zigpy-cli readme has not yet been updated to reflect all new changes:

https://github.com/zigpy/zigpy/wiki/OTA-Device-Firmware-Updates

https://github.com/zigpy/zigpy/wiki/OTA-Information-for-Manufacturers

https://github.com/zigpy/zigpy-cli/blob/dev/README.md#ota

My use case: I've had some issues recently with some IKEA devices behaving weirdly and it took me time to understand they were caused by recent OTA updates.

PS: Off-topic but FYI, I believe that the built-in support for IKEA Trådfri OTA updates has been temporarily disabled by default in the ZHA integration due to recent issues with those specific OTAU firmware images (so that still have to be enabled manually).