zigpy / zha-device-handlers

ZHA device handlers bridge the functionality gap created when manufacturers deviate from the ZCL specification, handling deviations and exceptions by parsing custom messages to and from Zigbee devices.
Apache License 2.0
705 stars 651 forks source link

[BUG] Ikea tradfri FYRTUR blinds reporting wrong battery % since latest firmware #2418

Closed alvinchen1 closed 6 months ago

alvinchen1 commented 1 year ago

Bug description

FYRTUR block-out roller blind with firmware 0x24040011, latest firmware deployed May 29, 2023 (https://ww8.ikea.com/ikeahomesmart/releasenotes/releasenotes.html) are reporting odd Battery %. Release notes include:

FYRTUR and KADRILJ roller blinds (24.4.11) Product ID: E1752, E1757 ◆ Optimised battery performance. ◆ Stability improvements.

image

Blinds which haven't gotten the new firmware (0x23088631) appear fine, but new firmware seems to now report ~2x the actual value.

image

Note in screenshot, Back Door Shade received firmware update 0x24040011 on 5/31/2023 at 8:55 AM and battery % doubled.

image

Another Example

image

Received firmware today, battery went from 44% to 100% ~2x.

image

Steps to reproduce

  1. Device receives firmware update 0x24040011
  2. Device's battery doubles.

Expected behavior

  1. Device receives firmware update 0.24040011
  2. Device's battery does not double? shrugs

Screenshots/Video

Screenshots/Video [Paste/upload your media here]

Device signature

Device signature ```json { "node_descriptor": "NodeDescriptor(logical_type=, complex_descriptor_available=0, user_descriptor_available=0, reserved=0, aps_flags=0, frequency_band=, mac_capability_flags=, manufacturer_code=4476, maximum_buffer_size=82, maximum_incoming_transfer_size=82, server_mask=11264, maximum_outgoing_transfer_size=82, descriptor_capability_field=, *allocate_address=True, *is_alternate_pan_coordinator=False, *is_coordinator=False, *is_end_device=True, *is_full_function_device=False, *is_mains_powered=False, *is_receiver_on_when_idle=False, *is_router=False, *is_security_capable=False)", "endpoints": { "1": { "profile_id": "0x0104", "device_type": "0x0202", "input_clusters": [ "0x0000", "0x0001", "0x0003", "0x0004", "0x0005", "0x0020", "0x0102", "0x1000", "0xfc7c" ], "output_clusters": [ "0x0019", "0x1000" ] } }, "manufacturer": "IKEA of Sweden", "model": "FYRTUR block-out roller blind", "class": "zhaquirks.ikea.blinds.IkeaTradfriRollerBlinds" } ```

Diagnostic information

Diagnostic information ```json [Paste the diagnostic information here] ```

Logs

Logs ```python { "home_assistant": { "installation_type": "Home Assistant OS", "version": "2023.5.4", "dev": false, "hassio": true, "virtualenv": false, "python_version": "3.10.11", "docker": true, "arch": "x86_64", "timezone": "America/New_York", "os_name": "Linux", "os_version": "6.1.29", "supervisor": "2023.04.1", "host_os": "Home Assistant OS 10.2", "docker_version": "23.0.6", "chassis": "embedded", "run_as_root": true }, "custom_components": { "edgeos": { "version": "2.0.28", "requirements": [ "aiohttp" ] }, "composite": { "version": "2.8.2", "requirements": [] }, "auto_backup": { "version": "1.3.2", "requirements": [] }, "sonos_cloud": { "version": "0.3.5", "requirements": [] }, "hacs": { "version": "1.32.1", "requirements": [ "aiogithubapi>=22.10.1" ] }, "garmin_connect": { "version": "0.2.16", "requirements": [ "garminconnect==0.1.55", "tzlocal" ] }, "webrtc": { "version": "v3.1.0", "requirements": [] }, "huesyncbox": { "version": "1.23.0", "requirements": [ "aiohuesyncbox==0.0.21" ] }, "nodered": { "version": "1.2.0", "requirements": [] }, "powercalc": { "version": "v1.6.5", "requirements": [ "numpy>=1.21.1" ] }, "emporia_vue": { "version": "0.8.1", "requirements": [ "pyemvue==0.16.2" ] } }, "integration_manifest": { "domain": "zha", "name": "Zigbee Home Automation", "after_dependencies": [ "onboarding", "usb" ], "codeowners": [ "@dmulcahey", "@adminiuga", "@puddly" ], "config_flow": true, "dependencies": [ "file_upload" ], "documentation": "https://www.home-assistant.io/integrations/zha", "iot_class": "local_polling", "loggers": [ "aiosqlite", "bellows", "crccheck", "pure_pcapy3", "zhaquirks", "zigpy", "zigpy_deconz", "zigpy_xbee", "zigpy_zigate", "zigpy_znp" ], "requirements": [ "bellows==0.35.5", "pyserial==3.5", "pyserial-asyncio==0.6", "zha-quirks==0.0.99", "zigpy-deconz==0.21.0", "zigpy==0.55.0", "zigpy-xbee==0.18.0", "zigpy-zigate==0.11.0", "zigpy-znp==0.11.1" ], "usb": [ { "vid": "10C4", "pid": "EA60", "description": "*2652*", "known_devices": [ "slae.sh cc2652rb stick" ] }, { "vid": "1A86", "pid": "55D4", "description": "*sonoff*plus*", "known_devices": [ "sonoff zigbee dongle plus v2" ] }, { "vid": "10C4", "pid": "EA60", "description": "*sonoff*plus*", "known_devices": [ "sonoff zigbee dongle plus" ] }, { "vid": "10C4", "pid": "EA60", "description": "*tubeszb*", "known_devices": [ "TubesZB Coordinator" ] }, { "vid": "1A86", "pid": "7523", "description": "*tubeszb*", "known_devices": [ "TubesZB Coordinator" ] }, { "vid": "1A86", "pid": "7523", "description": "*zigstar*", "known_devices": [ "ZigStar Coordinators" ] }, { "vid": "1CF1", "pid": "0030", "description": "*conbee*", "known_devices": [ "Conbee II" ] }, { "vid": "10C4", "pid": "8A2A", "description": "*zigbee*", "known_devices": [ "Nortek HUSBZB-1" ] }, { "vid": "0403", "pid": "6015", "description": "*zigate*", "known_devices": [ "ZiGate+" ] }, { "vid": "10C4", "pid": "EA60", "description": "*zigate*", "known_devices": [ "ZiGate" ] }, { "vid": "10C4", "pid": "8B34", "description": "*bv 2010/10*", "known_devices": [ "Bitron Video AV2010/10" ] } ], "zeroconf": [ { "type": "_esphomelib._tcp.local.", "name": "tube*" }, { "type": "_zigate-zigbee-gateway._tcp.local.", "name": "*zigate*" }, { "type": "_zigstar_gw._tcp.local.", "name": "*zigstar*" }, { "type": "_slzb-06._tcp.local.", "name": "slzb-06*" } ], "is_built_in": true }, "data": { "ieee": "**REDACTED**", "nwk": 16729, "manufacturer": "IKEA of Sweden", "model": "FYRTUR block-out roller blind", "name": "IKEA of Sweden FYRTUR block-out roller blind", "quirk_applied": true, "quirk_class": "zhaquirks.ikea.blinds.IkeaTradfriRollerBlinds", "manufacturer_code": 4476, "power_source": "Battery or Unknown", "lqi": 184, "rssi": -54, "last_seen": "2023-05-31T18:01:58", "available": true, "device_type": "EndDevice", "signature": { "node_descriptor": "NodeDescriptor(logical_type=, complex_descriptor_available=0, user_descriptor_available=0, reserved=0, aps_flags=0, frequency_band=, mac_capability_flags=, manufacturer_code=4476, maximum_buffer_size=82, maximum_incoming_transfer_size=82, server_mask=11264, maximum_outgoing_transfer_size=82, descriptor_capability_field=, *allocate_address=True, *is_alternate_pan_coordinator=False, *is_coordinator=False, *is_end_device=True, *is_full_function_device=False, *is_mains_powered=False, *is_receiver_on_when_idle=False, *is_router=False, *is_security_capable=False)", "endpoints": { "1": { "profile_id": "0x0104", "device_type": "0x0202", "input_clusters": [ "0x0000", "0x0001", "0x0003", "0x0004", "0x0005", "0x0020", "0x0102", "0x1000", "0xfc7c" ], "output_clusters": [ "0x0019", "0x1000" ] } }, "manufacturer": "IKEA of Sweden", "model": "FYRTUR block-out roller blind" }, "active_coordinator": false, "entities": [ { "entity_id": "button.dining_room_display_right_window_shade_identify", "name": "IKEA of Sweden FYRTUR block-out roller blind" }, { "entity_id": "cover.dining_room_display_right_window_shade", "name": "IKEA of Sweden FYRTUR block-out roller blind" }, { "entity_id": "sensor.dining_room_display_right_window_shade_battery", "name": "IKEA of Sweden FYRTUR block-out roller blind" } ], "neighbors": [], "routes": [], "endpoint_names": [ { "name": "WINDOW_COVERING_DEVICE" } ], "user_given_name": "Dining Room Display Right Window Shade", "device_reg_id": "0fd5dc9054d57bb37652924dda0da7ba", "area_id": "dining_room", "cluster_details": { "1": { "device_type": { "name": "WINDOW_COVERING_DEVICE", "id": 514 }, "profile_id": 260, "in_clusters": { "0x0000": { "endpoint_attribute": "basic", "attributes": { "0x0004": { "attribute_name": "manufacturer", "value": "IKEA of Sweden" }, "0x0005": { "attribute_name": "model", "value": "FYRTUR block-out roller blind" } }, "unsupported_attributes": {} }, "0x0001": { "endpoint_attribute": "power", "attributes": { "0x0020": { "attribute_name": "battery_voltage", "value": 71 }, "0x0021": { "attribute_name": "battery_percentage_remaining", "value": 172 } }, "unsupported_attributes": { "0x0031": { "attribute_name": "battery_size" }, "0x0033": { "attribute_name": "battery_quantity" } } }, "0x0003": { "endpoint_attribute": "identify", "attributes": {}, "unsupported_attributes": {} }, "0x0004": { "endpoint_attribute": "groups", "attributes": {}, "unsupported_attributes": {} }, "0x0005": { "endpoint_attribute": "scenes", "attributes": {}, "unsupported_attributes": {} }, "0x0020": { "endpoint_attribute": "poll_control", "attributes": { "0x0000": { "attribute_name": "checkin_interval", "value": 13200 } }, "unsupported_attributes": {} }, "0x0102": { "endpoint_attribute": "window_covering", "attributes": { "0x0008": { "attribute_name": "current_position_lift_percentage", "value": 100 } }, "unsupported_attributes": {} }, "0x1000": { "endpoint_attribute": "lightlink", "attributes": {}, "unsupported_attributes": {} }, "0xfc7c": { "endpoint_attribute": "manufacturer_specific", "attributes": {}, "unsupported_attributes": {} } }, "out_clusters": { "0x0019": { "endpoint_attribute": "ota", "attributes": {}, "unsupported_attributes": {} }, "0x1000": { "endpoint_attribute": "lightlink", "attributes": {}, "unsupported_attributes": {} } } } } } } ```

Additional information

No response

MattWestb commented 1 year ago

DIrigera have updating them for over one month ago and TF was rolling out many updates yesterday evening. Then IKEA is very Zigbee its only have battery fix in the quirk that was wrong but i fixed in the 24.X firmware. But IKEA have also changing the cluster config so the updated firmware is not loading the old quirk if being new added but is loading it if using the old settings. So deleting one device and wait one minute and adding it new so shall it being added without quirk and reporting the battery OK.

jasondefuria commented 1 year ago

I too have this issue. I have tried resetting/readding, but it stays with the percentage that is double of what I think it actually is.

MattWestb commented 1 year ago

Look if the quirk is being loaded for the device. If being loaded you is getting 200% battery. If not it shall working and normal and reporting 100% battery then its full.

alvinchen1 commented 1 year ago

Once all my blinds updated, I removed each, waited a minute or two before resetting (hold both buttons for 5 seconds until white LED started flashing) and added them back in. Each one came back with the right battery status.

jasondefuria commented 1 year ago

@alvinchen1 Did you remove them from home assistant via the GUI? Or did you just reset them (holding both buttons until the led comes on). Maybe not removing them was the issue on my end- I have a number of those blinds so I just did the 5 second button hold reset.

alvinchen1 commented 1 year ago

Yup! Remove device from HA via the GUI, wait a minute, then reset each, re-added each. Sorry that wasn't clear.

MattWestb commented 1 year ago

Its needed deleting it in ZHA and waiting one minute or restarting HA so the database have committing the deletion then resetting the device and adding it and is being done as one new device and reading the new configuration of it. If not deleting it in ZHA and waiting its being readed with the old config = battery not working OK.

jasondefuria commented 1 year ago

Yup, that definitely fixed it. Thanks!!!

oscar-b commented 1 year ago

@MattWestb Hey, I have the same issue with "TREDANSEN" (TREDANSEN block-out cellul blind) blinds, the DoublingPowerConfigurationCluster is being applied here: https://github.com/zigpy/zha-device-handlers/blob/dev/zhaquirks/ikea/blinds.py#L69

Is this IKEA following the Zigbee standard better so that we don't need this quirk at all anymore? I can't see anything else being fixed byt this quirk?

https://github.com/zigpy/zha-device-handlers/blob/dev/zhaquirks/__init__.py#L178

Re-read your comment, I think you mean that IKEA changed their cluster ID in the new firmware, so this quirk shouldn't be loaded for updated devices, but you need to re-add for the quirk to be removed. Correct? Will try straight away :)

oscar-b commented 1 year ago

@MattWestb It seems to work now (no quirk loaded), but not because IKEA changed their cluster ID, but because they added an output cluster: 0x03

{
  "node_descriptor": "NodeDescriptor(logical_type=<LogicalType.EndDevice: 2>, complex_descriptor_available=0, user_descriptor_available=0, reserved=0, aps_flags=0, frequency_band=<FrequencyBand.Freq2400MHz: 8>, mac_capability_flags=<MACCapabilityFlags.AllocateAddress: 128>, manufacturer_code=4476, maximum_buffer_size=82, maximum_incoming_transfer_size=82, server_mask=11264, maximum_outgoing_transfer_size=82, descriptor_capability_field=<DescriptorCapability.NONE: 0>, *allocate_address=True, *is_alternate_pan_coordinator=False, *is_coordinator=False, *is_end_device=True, *is_full_function_device=False, *is_mains_powered=False, *is_receiver_on_when_idle=False, *is_router=False, *is_security_capable=False)",
  "endpoints": {
    "1": {
      "profile_id": "0x0104",
      "device_type": "0x0202",
      "input_clusters": [
        "0x0000",
        "0x0001",
        "0x0003",
        "0x0004",
        "0x0005",
        "0x0020",
        "0x0102",
        "0x1000",
        "0xfc7c"
      ],
      "output_clusters": [
        "0x0003",
        "0x0019",
        "0x1000"
      ]
    }
  },
  "manufacturer": "IKEA of Sweden",
  "model": "TREDANSEN block-out cellul blind",
  "class": "zigpy.device.Device"
}
MattWestb commented 1 year ago

Yea its looks like my is doing and its not loading the old quirks and then the battery readings is being OK that is fixed in the latest firmware. Normally is needed deleting the device and waiting one minute or restarting HA and adding it for the quirk matching working OK after firmware update.

oscar-b commented 1 year ago

It is behaving a little weirdly now though, it doesn't seem to report that it stops moving after it stops (when using cover.set_cover_position)..

MattWestb commented 1 year ago

I have not testing it but it can being some changes in the firmware that is doing that. Try doing one "reconfigure device" from the device card and see if it configuring reporting on the "WindowCovering" cluster or not. Older firmware was not needing some config then was automatic reporting to the coordinator but i have not sniffing how the new 24.X firmware is being done in TF and Dirigear then its being paired.

oscar-b commented 1 year ago

@MattWestb You mean this?

image
oscar-b commented 1 year ago

Doesn't it need another attribute to know if it's moving or not?

MattWestb commented 1 year ago

The config iscurrent_position_lift_percentage: 0/900/1 = min / may time / Change. So if the position is changed by 1 its timidity reported or if not changed its reported after the last report + 900 seconds (i think or 1/4 or 1/10). If you using one remote and changing the blinds is it reported in the device card ?

oscar-b commented 1 year ago

@MattWestb I'm opening/closing using the UI in Home Assistant, dragging to a percentage (triggering cover.set_cover_position). The percentage is reported when opening/closing, then the percentage stops but it still says opening/closing and not open/close.

It also seems to be a little wonky when trying to issue a new command when it's in this state, it sometimes isn't accepted (I guess because HA thinks it's moving still).

I don't have the remote connected.

MattWestb commented 1 year ago

I was testing the slider in the GUI and its working OK then sliding the GUT is reporting back the % then its working and also then it sopped the final %. Using the up, stop and down its also working OK and up is graying out the its 100% open and the down then its 100% closed and the stop is then its not moving.

My RCP test is running: Home Assistant 2023.6.3 Supervisor 2023.06.4 Frontend 20230608.0 - latest

And the device: Device info TREDANSEN block-out cellul blind by IKEA of Sweden Connected via Billy RCP 4.2.1 RK3318 Firmware: 0x24040011

Edit: I was deleting and adding it new then i was not remember if i was doing it after the last firmware update but its still working OK.

oscar-b commented 1 year ago
TREDANSEN block-out cellul blind
Firmware: 0x24040011
Home Assistant 2023.6.3
Texas Instruments CC1352/CC2652, Z-Stack 3.30+ (build 20211217)

cover.close_cover and cover.open_cover works fine, the blinds move, reports properly in HA and stops moving.

cover.set_cover_position works, but it just keeps saying "Opening - 10%", instead of "Open - 10%". Manually doing cover.stop_cover "resolves" the issue.

How does HA know when the blinds stop moving? Which attribute in ZHA is used for this?

Update: https://github.com/home-assistant/core/blob/master/homeassistant/components/zha/cover.py Will see what I can figure out from this.

MattWestb commented 1 year ago

@TheJulianJES and @dmulcahey do some of you having one blinds for testing ? Its start looking its one problem in the frontend then the reporting from the device looks working very good but the GUI is not doing all right.

Thanks in advance !!

oscar-b commented 1 year ago

I haven't really looked at HA code that much before, and I'm no Python developer, but I tried looking through the code a bit and have some questions:

  1. Why is get_attribute_value("current_position_lift_percentage") done in two different places, both https://github.com/home-assistant/core/blob/dev/homeassistant/components/zha/core/cluster_handlers/closures.py#L134-L146 and https://github.com/home-assistant/core/blob/dev/homeassistant/components/zha/cover.py#L172-L178 ?
  2. Why isn't self.async_write_ha_state() called in either of those places, is it done implicitly by HA?
  3. When does HA call async_update()? If I trigger service: homeassistant.update_entity it's called, and the state of the cover is updated properly:
2023-07-04 00:28:57.572 DEBUG (MainThread) [homeassistant.components.zha.core.cluster_handlers] [0x0110:1:0x0102]: Attribute report 'Window Covering'[current_position_lift_percentage] = 95
2023-07-04 00:28:57.572 DEBUG (MainThread) [homeassistant.components.zha.cover] setting position: 95
2023-07-04 00:28:57.573 DEBUG (MainThread) [homeassistant.components.zha.cover] setting position: 95
2023-07-04 00:28:57.573 DEBUG (MainThread) [homeassistant.components.zha.cover] polling current state
2023-07-04 00:28:57.573 DEBUG (MainThread) [homeassistant.components.zha.cover] read pos=95
  1. How is the state supposed to go back to STATE_OPEN or STATE_CLOSED? The only places I see that's being done is when async_stop_cover() is called (I assume this is an explicit stop call from HA) or when the position is updated to 0 or 100 (in async_set_position()). Anything in between there seems like it should be handled by async_update() / async_get_state() (see 2 and 3). Shouldn't there be anything in async_set_position() checking the position against the desired position?
oscar-b commented 1 year ago

Its start looking its one problem in the frontend then the reporting from the device looks working very good but the GUI is not doing all right.

The state of the entity is stuck in opening/closing, so I don't see this being a frontend issue at all.

tom3q commented 1 year ago

Removing the blinds from HA completely and re-adding them worked for me, thanks!

@oscar-b , please make sure yours has indeed the new firmware (24.4.x). It solved similar issue to me - although I needed to hard-reset it by removing the battery for a few seconds.

MattWestb commented 1 year ago

Also need waiting one minute after deleting it before adding it new so the database is committed or it using the old data saved in it => readding not as new. Also restarting HA is working well.

oscar-b commented 1 year ago

@MattWestb I will try this again, but I still fail to see where, in the code, the state is supposed to be set back to STATE_OPEN / STATE_CLOSED.

oscar-b commented 1 year ago

@tom3q If you use the UI to set the blinds to a particular position, does it immediately get the state Open / Closed when it arrives at the position and stops?

oscar-b commented 1 year ago

@MattWestb I will try this again, but I still fail to see where, in the code, the state is supposed to be set back to STATE_OPEN / STATE_CLOSED.

Did a complete remove and re-add, without any difference in behaviour. Still works fine, but keeps "Closing" or "Opening" even after reaching the desired position (as I'd expect after looking at the code).

tom3q commented 1 year ago

Ah, I see what you mean. Indeed if I use the position slider, it gets stuck in opening or closing state.

I usually use the open and close buttons and pretty much always go between 0 and 100 only and for those case it switched to opened and closed states correctly.

2023年7月8日(土) 17:03 Oscar Bolmsten @.***>:

@MattWestb https://github.com/MattWestb I will try this again, but I still fail to see where, in the code, the state is supposed to be set back to STATE_OPEN / STATE_CLOSED.

Did a complete remove and re-add, without any difference in behaviour. Still works fine, but keeps "Closing" or "Opening" even after reaching the desired position (as I'd expect after looking at the code).

— Reply to this email directly, view it on GitHub https://github.com/zigpy/zha-device-handlers/issues/2418#issuecomment-1626903312, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADBVWYQQPYCONNXWBM7IHLXPEH4FANCNFSM6AAAAAAYWCKKUM . You are receiving this because you were mentioned.Message ID: @.***>

Triberg commented 1 year ago

I removed the blinds+remote from the GUI, factory reset them, and re-paired them. After updating my scripts to stop using position (and use Open Cover and Close Cover instead), I'm still seeing the blinds stuck in the "opening" state. And my remote battery isn't reporting anything.

chicagoandy commented 1 year ago

My blinds are very high and out of reach so I would prefer not to remove them from HomeAssistant as adding them requires button-pushes. Is there another fix that doesn't involve removing them? Or a way to fix this without having to push buttons on the blinds?

alvinchen1 commented 9 months ago

Hi, wanted to revive this slightly. So when I originally removed and readded all my devices to resolve this battery issue, it worked, but I did notice as others have mentioned that the additional output cluster caused all the devices to lose the quirk. After living with this for several months (and everything seemed fine), I noticed that one of the things the quirk does is when it opens the shade, it will open it completely. In the past few months, I've noticed my shades leaving anywhere from an inch to three inches of "open" even when the device claims to be 100% open. This "inch" gap also results in when completely closed, that it droops below the fully closed location which I had carefully set across all my shades. I've been able to get around this when replacing batteries because that will cause the shade to reset and close the "inch".

I've attempted to modify the quirk so that the shades open completely (and close at the right spot too).

See https://github.com/zigpy/zha-device-handlers/discussions/2754

If you'd like to try my custom quirk on your shades (still a work in progress before I PR and working on one issue which I could use some help with), do the following:

Using File Editor/Core Configurator addon, create a folder called custom_zha_quirks Download my quirk into this folder (wget https://raw.githubusercontent.com/alvinchen1/zha-device-handlers/dev/zhaquirks/ikea/blinds.py)

In configuration.yaml add

zha:
  custom_quirks_path: /config/custom_zha_quirks
  database_path: /config/zigbee.db
  enable_quirks: true

Reboot HA. Look at the properties of your device and see if you see the quirk

image

TheJulianJES commented 6 months ago

This issue should be fixed in HA Core 2024.2.0 (currently in beta). Relevant PR:

If you still see new "double battery percentage" reports, wait a day. You can also try to press a button on the device or manually read the sw_build_id attriubte. Re-pairing/reconfiguring should also work.

If it's not fixed for you, please create a new issue and provide a new diagnostics file.