search

pvvx / ZigbeeTLc

Custom firmware for Zigbee 3.0 IoT devices on the TLSR825x chip
Other
404 stars 23 forks source link

OTA update in ZHA #7

Open pvvx opened 1 year ago

pvvx commented 1 year ago

OTA update in ZHA.

The "configuration.yaml" file must have a directory assigned to the OTA files and the "ZHA Toolkit" installed.

For example, the directory name will be: “/zigbee_ota”

zha:
  zigpy_config:
    ota:
      enabled: true
      # Download the index over HTTPS
      z2m_remote_index: https://raw.githubusercontent.com/Koenkk/zigbee-OTA/master/index.json
      # Or, load the index offline
      z2m_local_index: ./zigbee_ota/index.json
      allow_advanced_ota_dir: I understand I can *destroy* my devices by enabling OTA
        updates from files. Some OTA updates can be mistakenly applied to the
        wrong device, breaking it. I am consciously using this at my own risk.
      advanced_ota_dir: ./zigbee_ota
      otau_directory: ./zigbee_ota
      ikea_provider: false
      ledvance_provider: false
      salus_provider: false
      sonoff_provider: false
      inovelli_provider: false
      thirdreality_provider: false

Place the new FW version files "*.zigbee" or "index.json" from bin into this directory ("./zigbee_ota").

Restart ZHA (preferably the entire Home Assistant).

Update in new versions of Home Assistant:

image

image

Update in ZHA Toolkit:

Doesn't work in the new version of ZHA!

https://github.com/mdeweerd/zha-toolkit#ota_notify---downloadtrigger-device-fw-update

Go to “Developer tools” -> “SERVICES” and select “ZHA Toolkit: Trigger Device's Firmware Update”.

image

Select “Device Reference” and click “CALL SERVICE”.

After some time, 15..20 minutes, the firmware number will change:

image

To restore to BLE, use the Zigbee OTA file with the same number device from: https://github.com/pvvx/ATC_MiThermometer/tree/master/zigbee_ota

After running Zigbee OTA, be sure to delete this file or files from the update directories for Zigbee OTA and completely restart z2m/zha. Otherwise, with the new Zigbee firmware, this update in BLE will work again.

bs76 commented 1 year ago

Tried to follow restoring BLE but stumbled on these issues:

Any advice ?

pvvx commented 1 year ago

Download the file correctly. Not as “blob”, but as “raw”. https://github.com/pvvx/ZigbeeTLc/raw/master/bin/1141-020a-01123001-Z03MMC.zigbee

bs76 commented 1 year ago

D'oh ! Embarrasing mistake on my side. Thank you, succesfully reflashed

breaver commented 1 year ago

OTA update in ZHA.

Do you have such an walkthrough for zigbee2mqtt?

pvvx commented 1 year ago

Do you have such an walkthrough for zigbee2mqtt?

https://www.zigbee2mqtt.io/guide/usage/ota_updates.html#local-ota-index-and-firmware-files

breaver commented 12 months ago

I followed the procedure but still not able to do the OTA update back to BLE. Z2M keeps telling me Failed to check if update available for 'TF Sensor ' (No image available for imageType '522')

CLARENNE-Q commented 11 months ago

Hello,

If you have HomeAssistant OS, the configuration file must be like this:

      otau_directory: ./zigpy_ota
pvvx commented 11 months ago

https://github.com/zigpy/zigpy/wiki/OTA-Device-Firmware-Updates#enabling-ota-updates

onurunaldi commented 11 months ago

Hello,

If you have HomeAssistant OS, the configuration file must be like this:

      otau_directory: ./zigpy_ota

This solved my problem and this trick is not written anywhere. Thanks so much!

rwronski commented 11 months ago

Unfortunately, OTA in zigbee2mqtt doesn't work for me either (HA OS). I've followed the manual, created override OTA index, tried many things, but it just doesn't work. If anyone has any other experience, please share.

Only error I get is 'Failed to check if update available for 'Termometr 2' (No image available for imageType '522')' - it seems like z2m is just ignoring my local OTA index.

rwronski commented 11 months ago

Ok I've managed to make a progress - you don't add 'zigbee_ota_override_index_location' to the configuration.yaml file manually, but go to 'Settings' -> 'OTA Updates' -> fill in the 'OTA index override file name' field. And suddenly z2m sees it, even though the configuration.yaml looks the same as if I'd edited it manually.

Neo2SHYAlien commented 11 months ago

@rwronski try this guide. Personally I didn't validate it

khayamgondal commented 9 months ago

@pvvx where is the BLE .zigbee file for tuya TS0201?

djbuldog commented 9 months ago

Hi, I have LYWSD03MMC with devbis fw 1.1.0 and I would like to migrate to ZigbeTLc fw or back to BT. I followed guides for Z2M and ZHA but without success. For example ZHA.. I added following to configuration.yaml:

zha:
  zigpy_config:
    ota:
      otau_directory: ./zigpy_ota

zha_toolkit:

the folder /root/config/zigpy_ota contains:

-rw-r--r--    1 root     root        128034 Feb 17 00:25 1141-0203-09993001-Z03MMC_v0120.zigbee
-rw-r--r--    1 root     root         82674 Feb 17 00:29 1141-0203-99993001-ATC_v46.zigbee

I called ZHA Toolkit: Trigger Device's Firmware Update service. The log contains:

2024-02-17 01:17:36.680 INFO (MainThread) [custom_components.zha_toolkit] Running ZHA Toolkit service: <ServiceCall zha_toolkit.ota_notify (c:01HPT6SXX821EAJN6CBNR4XVQS): ieee=update.xiaomi_lywsd03mmc_firmware>
2024-02-17 01:17:36.861 DEBUG (MainThread) [zigpy.application] Feeding watchdog
2024-02-17 01:17:36.939 DEBUG (MainThread) [zigpy.ota.provider] Inovelli: Finished downloading firmware update list
2024-02-17 01:17:37.170 DEBUG (MainThread) [zigpy.ota.provider] Ledvance: Finished downloading firmware update list
2024-02-17 01:17:37.616 DEBUG (MainThread) [zigpy.ota.provider] Salus: Finished downloading firmware update list
2024-02-17 01:17:37.685 DEBUG (MainThread) [zigpy.ota.provider] Sonoff: Finished downloading firmware update list
2024-02-17 01:17:38.169 DEBUG (MainThread) [zigpy.ota.provider] ThirdReality: Finished downloading firmware update list
2024-02-17 01:17:38.169 DEBUG (MainThread) [zigpy.device] [0xec8b] Extending timeout for 0x04 request
2024-02-17 01:17:44.143 DEBUG (MainThread) [zigpy.application] Received a packet: ZigbeePacket(timestamp=datetime.datetime(2024, 2, 17, 0, 17, 44, 143898, tzinfo=datetime.timezone.utc), src=AddrModeAddress(addr_mode=<AddrMode.NWK: 2>, address=0xEC8B), src_ep=0, dst=AddrModeAddress(addr_mode=<AddrMode.NWK: 2>, address=0x0000), dst_ep=0, source_route=None, extended_timeout=False, tsn=209, profile_id=0, cluster_id=32801, data=Serialized[b'\x04\x00'], tx_options=<TransmitOptions.NONE: 0>, radius=0, non_member_radius=0, lqi=244, rssi=-39)
2024-02-17 01:17:44.144 DEBUG (MainThread) [zigpy.zcl] [0xEC8B:1:0x0000] Sending request header: ZCLHeader(frame_control=FrameControl<0x00>(frame_type=<FrameType.GLOBAL_COMMAND: 0>, is_manufacturer_specific=False, direction=<Direction.Client_to_Server: 0>, disable_default_response=0, reserved=0, *is_cluster=False, *is_general=True), tsn=5, command_id=<GeneralCommand.Configure_Reporting: 6>, *direction=<Direction.Client_to_Server: 0>)
2024-02-17 01:17:44.144 DEBUG (MainThread) [zigpy.zcl] [0xEC8B:1:0x0000] Sending request: Configure_Reporting(config_records=[AttributeReportingConfig(direction=0, attrid=0x4000, datatype=66, min_interval=0, max_interval=1800, reportable_change=1)])
2024-02-17 01:17:44.144 DEBUG (MainThread) [zigpy.device] [0xec8b] Extending timeout for 0x05 request
2024-02-17 01:17:44.690 DEBUG (MainThread) [zigpy.application] Received a packet: ZigbeePacket(timestamp=datetime.datetime(2024, 2, 17, 0, 17, 44, 690278, tzinfo=datetime.timezone.utc), src=AddrModeAddress(addr_mode=<AddrMode.NWK: 2>, address=0xEC8B), src_ep=1, dst=AddrModeAddress(addr_mode=<AddrMode.NWK: 2>, address=0x0000), dst_ep=1, source_route=None, extended_timeout=False, tsn=210, profile_id=260, cluster_id=0, data=Serialized[b'\x18\x05\x07\x86\x00\x00@'], tx_options=<TransmitOptions.NONE: 0>, radius=0, non_member_radius=0, lqi=244, rssi=-39)
2024-02-17 01:17:44.690 DEBUG (MainThread) [zigpy.zcl] [0xEC8B:1:0x0000] Received ZCL frame: b'\x18\x05\x07\x86\x00\x00@'
2024-02-17 01:17:44.690 DEBUG (MainThread) [zigpy.zcl] [0xEC8B:1:0x0000] Decoded ZCL frame header: ZCLHeader(frame_control=FrameControl<0x18>(frame_type=<FrameType.GLOBAL_COMMAND: 0>, is_manufacturer_specific=0, direction=<Direction.Server_to_Client: 1>, disable_default_response=1, reserved=0, *is_cluster=False, *is_general=True), tsn=5, command_id=7, *direction=<Direction.Server_to_Client: 1>)
2024-02-17 01:17:44.690 DEBUG (MainThread) [zigpy.zcl] [0xEC8B:1:0x0000] Decoded ZCL frame: Basic:Configure_Reporting_rsp(status_records=[ConfigureReportingResponseRecord(status=<Status.UNSUPPORTED_ATTRIBUTE: 134>, direction=<ReportingDirection.SendReports: 0>, attrid=16384)])
2024-02-17 01:17:44.691 DEBUG (MainThread) [zigpy.zcl] [0xEC8B:1:0x0019] Sending reply header: ZCLHeader(frame_control=FrameControl<0x19>(frame_type=<FrameType.CLUSTER_COMMAND: 1>, is_manufacturer_specific=False, direction=<Direction.Server_to_Client: 1>, disable_default_response=1, reserved=0, *is_cluster=True, *is_general=False), tsn=6, command_id=0, *direction=<Direction.Server_to_Client: 1>)
2024-02-17 01:17:44.691 DEBUG (MainThread) [zigpy.zcl] [0xEC8B:1:0x0019] Sending reply: ImageNotifyCommand(payload_type=<PayloadType.QueryJitter: 0>, query_jitter=100)
2024-02-17 01:17:45.207 INFO (MainThread) [homeassistant.helpers.script.websocket_api_script] websocket_api script: Stop script sequence: done
2024-02-17 01:17:45.221 DEBUG (MainThread) [zigpy.application] Received a packet: ZigbeePacket(timestamp=datetime.datetime(2024, 2, 17, 0, 17, 45, 221578, tzinfo=datetime.timezone.utc), src=AddrModeAddress(addr_mode=<AddrMode.NWK: 2>, address=0xEC8B), src_ep=1, dst=AddrModeAddress(addr_mode=<AddrMode.NWK: 2>, address=0x0000), dst_ep=1, source_route=None, extended_timeout=False, tsn=211, profile_id=260, cluster_id=25, data=Serialized[b'\x01\xb0\x01\x00\x15\xdb\x03\x02\x010\x00\x11'], tx_options=<TransmitOptions.NONE: 0>, radius=0, non_member_radius=0, lqi=244, rssi=-39)
2024-02-17 01:17:45.221 DEBUG (MainThread) [zigpy.zcl] [0xEC8B:1:0x0019] Received ZCL frame: b'\x01\xb0\x01\x00\x15\xdb\x03\x02\x010\x00\x11'
2024-02-17 01:17:45.221 DEBUG (MainThread) [zigpy.zcl] [0xEC8B:1:0x0019] Decoded ZCL frame header: ZCLHeader(frame_control=FrameControl<0x01>(frame_type=<FrameType.CLUSTER_COMMAND: 1>, is_manufacturer_specific=0, direction=<Direction.Client_to_Server: 0>, disable_default_response=0, reserved=0, *is_cluster=True, *is_general=False), tsn=176, command_id=1, *direction=<Direction.Client_to_Server: 0>)
2024-02-17 01:17:45.222 DEBUG (MainThread) [zigpy.zcl] [0xEC8B:1:0x0019] Decoded ZCL frame: Ota:QueryNextImageCommand(field_control=<FieldControl: 0>, manufacturer_code=56085, image_type=515, current_file_version=285224961)
2024-02-17 01:17:45.222 DEBUG (MainThread) [zigpy.zcl] [0xEC8B:1:0x0019] Received command 0x01 (TSN 176): QueryNextImageCommand(field_control=<FieldControl: 0>, manufacturer_code=56085, image_type=515, current_file_version=285224961)
2024-02-17 01:17:45.222 DEBUG (MainThread) [zigpy.zcl] [0xEC8B:1:0x0019] OTA query_next_image handler for 'Xiaomi LYWSD03MMC': field_control=0, manufacturer_id=56085, image_type=515, current_file_version=285224961, hardware_version=None, model='LYWSD03MMC'
2024-02-17 01:17:45.222 DEBUG (MainThread) [zigpy.zcl] [0xEC8B:1:0x0019] No OTA image is available

there is one interesting message... I do not know what it means or if it is related to my issue... 

2024-02-17 00:34:06.142 INFO (MainThread) [zigpy.ota] OTA file providers are currently disabled

Core 2024.2.2 Supervisor 2024.01.1 Operating System 11.5 Frontend 20240207.1

It looks like the local OTA files are ignored... Can you see anything wrong? Any ideas what to try next?

robvanoostenrijk commented 9 months ago

zigpy has disabled local providers because many people are breaking their devices with wrong OTA updates.

The commit that does that is here: https://github.com/zigpy/zigpy/commit/6aeb8a6816585a4bda67e9ea1b125568c18fa702

You can still do OTA updates using ZHA with the following yaml configuration: (Note the new allow_file_providers key and its value)

zha:
  zigpy_config:
    ota:
      allow_file_providers: "I understand I can *destroy* my devices by enabling OTA updates from files. Some OTA updates can be mistakenly applied to the wrong device, breaking it. I am consciously using this at my own risk."
      ikea_provider: true
      inovelli_provider: false
      ledvance_provider: false
      otau_directory: ./zigbee_ota
      salus_provider: false
      sonoff_provider: false
      thirdreality_provider: false
ckpt-martin commented 8 months ago

After flashing the Zigbee firmware, is there a way to tell exactly which image was installed? I am unsure whether the image from pvvx or devbis were installed. I have followed everything above, including the new allow_file_providers line and have download the ATC 4.7 files for restoring BT as raw and not blob. I uploaded both the 0203 and 020a images into the zigpy_ota folder. Restarted HA, initiated the OTA image_notify command. But nothing happens. My devices always say firmware is up-to-date and the debug log says no compatible images found. My devices are on Firmware 0x01203001. I just have one that constantly loses connectivity. So, I wanted to restore the BT image to start over, just in case something went wrong during flashing. If that doesn't help, then I assume a faulty device. In which case I would also have to restore BT image to return it. Is there something else that needs to be done to allow ZHA to detect the correct firmware file under the zigpy_ota folder?

HAOS Core: 2024.3.1 Supervisor: 2024.03.0 OS: 12.1

pvvx commented 8 months ago

https://github.com/pvvx/ZigbeeTLc/issues/7#issue-2000695703

Zigbee->BLE Files https://github.com/pvvx/ATC_MiThermometer/blob/master/zigbee_ota/1141-0203-99993001-ATC_v47.zigbee https://github.com/pvvx/ATC_MiThermometer/blob/master/zigbee_ota/1141-020a-99993001-ATC_v47.zigbee

zha:
  zigpy_config:
    ota:
      allow_file_providers: "I understand I can *destroy* my devices by enabling OTA updates from files. Some OTA updates can be mistakenly applied to the wrong device, breaking it. I am consciously using this at my own risk."
      otau_directory: ./zigbee_ota

Have you tried using "reconfigure"? image image

Devices sometimes fall off if the coordinator is turned off for a long time (days) and the devices work through some Tuya Zigbee routers.

ckpt-martin commented 8 months ago

#7 (comment)

Zigbee->BLE Files https://github.com/pvvx/ATC_MiThermometer/blob/master/zigbee_ota/1141-0203-99993001-ATC_v47.zigbee https://github.com/pvvx/ATC_MiThermometer/blob/master/zigbee_ota/1141-020a-99993001-ATC_v47.zigbee


zha:
  zigpy_config:
    ota:
      allow_file_providers: "I understand I can *destroy* my devices by enabling OTA updates from files. Some OTA updates can be mistakenly applied to the wrong device, breaking it. I am consciously using this at my own risk."
      otau_directory: ./zigbee_ota

I had that setup. But you might want to modify your text above that you copied here. The directory that is referenced is "zigpy_ota", but in your example config you have "zigbee_ota". I suppose it doesn't matter as long as config and actual directory name are the same. ;-) And maybe your links to images should be modified, replacing "blob" with "raw" to make sure people download the raw version. I just tried the reconfigure option (after finally successfully pairing again) and it shows the same as your screenshots except the TemperatureMeasurement has an "X" next to the measured_value. :-(

maikvitesse commented 3 months ago

Just got it working thanks to this and this.

My working code now:

zha:
  zigpy_config:
    ota:
      allow_advanced_ota_dir:
        I understand I can *destroy* my devices by enabling OTA
        updates from files. Some OTA updates can be mistakenly applied to the
        wrong device, breaking it. I am consciously using this at my own risk.
      advanced_ota_dir: /config/zigpy_ota
cfulton commented 3 months ago

Newer working config

zha:
  zigpy_config:
    ota:
      extra_providers:
        - type: advanced
          warning: I understand I can *destroy* my devices by enabling OTA updates from
            files. Some OTA updates can be mistakenly applied to the wrong
            device, breaking it. I am consciously using this at my own risk.
          path: /config/zigpy_ota
fmauNeko commented 2 months ago

Even simpler

zha:
  zigpy_config:
    ota:
      extra_providers:
        - type: z2m
          url: https://raw.githubusercontent.com/pvvx/ZigbeeTLc/refs/heads/master/bin/index.json
michi2005 commented 2 months ago

Dear all

I tried a merge of the last 2 options but without success.

zha:
  zigpy_config:
    ota:
      extra_providers:
        - type: z2m
          warning: I understand I can *destroy* my devices by enabling OTA updates from
            files. Some OTA updates can be mistakenly applied to the wrong
            device, breaking it. I am consciously using this at my own risk.
          url: https://raw.githubusercontent.com/pvvx/ZigbeeTLc/refs/heads/master/bin/index.json

After that I get this error in ZHA:

Error setting up entry Sonoff Zigbee 3.0 USB Dongle Plus - Sonoff Zigbee 3.0 USB Dongle Plus for zha
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 604, in async_setup
    result = await component.async_setup_entry(hass, self)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/zha/__init__.py", line 132, in async_setup_entry
    async with radio_mgr.connect_zigpy_app() as app:
  File "/usr/local/lib/python3.12/contextlib.py", line 210, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/zha/radio_manager.py", line 182, in connect_zigpy_app
    app = await self.radio_type.controller.new(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/zigpy/application.py", line 251, in new
    app = cls(config)
          ^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/zigpy_znp/zigbee/application.py", line 81, in __init__
    super().__init__(config=config)
  File "/usr/local/lib/python3.12/site-packages/zigpy/application.py", line 68, in __init__
    self._config = self.SCHEMA(config)
                   ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/voluptuous/schema_builder.py", line 205, in __call__
    return self._compiled([], data)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/voluptuous/schema_builder.py", line 549, in validate_dict
    return base_validate(path, data.items(), out)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/voluptuous/schema_builder.py", line 382, in validate_mapping
    raise er.MultipleInvalid(errors)
voluptuous.error.MultipleInvalid: extra keys not allowed @ data['ota']['extra_providers'][0]['warning']
arthurkowalsky commented 1 month ago

If you're using Zigbee2MQTT and want to update your devices from devbis firmware to ZigbeeTLc using Over-The-Air (OTA) updates, follow these simple steps.

Prerequisites

Step-by-Step Guide

1. Download the Firmware File

2. Place the Firmware File in the Zigbee2MQTT Directory

3. Create a Custom OTA Index File

4. Configure Zigbee2Mqtt to Use the Custom OTA Index

image

5. Perform the OTA Update

image

6. Clean Up After the Update

Important Notes

sebgus commented 1 month ago

How can I reflash the OTA image or downgrade it?

I've managed to upgrade the firmware of a TS0201-TZ3000 to '0x01233001' (reported in ZHA), but the temperature reading reports -327.7 degrees C and the humidity is 655.4%. So something has gone wrong somewhere...

Regressor commented 1 month ago

If you're using Zigbee2MQTT and want to update your devices from devbis firmware to ZigbeeTLc using Over-The-Air (OTA) updates, follow these simple steps.

This should be in main readme! Thank you!

satmandu commented 1 month ago

[Edit: Solved.]

I'm trying this method to update from devbis firmware to ZigbeeTLc and I get this message:

Failed to check if update available for '0xa4c1385f065ca6c7' (Attempt to access memory outside buffer bounds)

Am I missing something?

Firmware build date
    20240204
Firmware version
    3001-1100
Manufacturer
    [Custom devices (DiY)](https://www.zigbee2mqtt.io/supported-devices/#v=Custom%20devices%20(DiY))
Model
    [LYWSD03MMC](https://www.zigbee2mqtt.io/devices/LYWSD03MMC.html#custom%2520devices%2520(diy)-lywsd03mmc)

I'm trying to update with db15-0203-79993001-Z03MMC_v0122.zigbee

End of /homeassistant/zigbee2mqtt/configuration.yaml:

ota:
  zigbee_ota_override_index_location: my_index.json

/homeassistant/zigbee2mqtt/my_index.json:

[
    {
        "url": "db15-0203-79993001-Z03MMC_v0122.zigbee",
        "force": true
    }
]

Edit: A reminder to all to actually download the RAW firmware links, not the links that connect straight to github pages!

(My update has now started... but it will take 23 minutes...)