zigpy / zha-device-handlers

ZHA device handlers bridge the functionality gap created when manufacturers deviate from the ZCL specification, handling deviations and exceptions by parsing custom messages to and from Zigbee devices.
Apache License 2.0
752 stars 688 forks source link

[BUG] Aqara plug (lumi.plug.maeu01) generates errors post-2022.2 #1337

Closed tetele closed 2 years ago

tetele commented 2 years ago

Describe the bug I use this plug with HA 2022.2.3, where it's been updated to use the quirk for lumi.plug.mmeu01 after this pull.

There are errors popping up in the log after this update.

Logger: homeassistant.util.logging
Source: util/logging.py:105
First occurred: 4:34:56 PM (16 occurrences)
Last logged: 4:55:26 PM

Exception in async_state_changed when dispatching 'LUMI lumi.plug.maeu01_54:ef:44:10:00:0e:52:9d_available_entity': () Traceback (most recent call last): 
 File  "/usr/src/homeassistant/homeassistant/components/zha/entity.py", line 107, in async_state_changed self.async_write_ha_state() 
 File  "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 530, in async_write_ha_state self._async_write_ha_state() 
 File  "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 563, in _async_write_ha_state state = self._stringify_state() 
 File  "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 536, in _stringify_state if (state := self.state) is None: 
 File  "/usr/src/homeassistant/homeassistant/components/sensor/__init__.py", line 371, in state value = self.native_value 
 File  "/usr/src/homeassistant/homeassistant/components/zha/sensor.py", line 175, in native_value return self.formatter(raw_state) 
 File  "/usr/src/homeassistant/homeassistant/components/zha/sensor.py", line 472, in formatter return self._channel.summa_formatter(value) 
 File  "/usr/src/homeassistant/homeassistant/components/zha/core/channels/smartenergy.py", line 196, in _formatter_function return self._summa_format.format(value).lstrip() AttributeError: 'NoneType' object has no attribute 'format'

To Reproduce Steps to reproduce the behavior: unknown

Additional context

{
  "node_descriptor": "NodeDescriptor(logical_type=<LogicalType.Router: 1>, complex_descriptor_available=0, user_descriptor_available=0, reserved=0, aps_flags=0, frequency_band=<FrequencyBand.Freq2400MHz: 8>, mac_capability_flags=<MACCapabilityFlags.AllocateAddress|RxOnWhenIdle|MainsPowered|FullFunctionDevice: 142>, manufacturer_code=4447, maximum_buffer_size=127, maximum_incoming_transfer_size=100, server_mask=11264, maximum_outgoing_transfer_size=100, descriptor_capability_field=<DescriptorCapability.NONE: 0>, *allocate_address=True, *is_alternate_pan_coordinator=False, *is_coordinator=False, *is_end_device=False, *is_full_function_device=True, *is_mains_powered=True, *is_receiver_on_when_idle=True, *is_router=True, *is_security_capable=False)",
  "endpoints": {
    "1": {
      "profile_id": 260,
      "device_type": "0x0051",
      "in_clusters": [
        "0x0000",
        "0x0002",
        "0x0003",
        "0x0004",
        "0x0005",
        "0x0006",
        "0x0009",
        "0x0702",
        "0x0b04"
      ],
      "out_clusters": [
        "0x000a",
        "0x0019"
      ]
    },
    "21": {
      "profile_id": 260,
      "device_type": "0x0009",
      "in_clusters": [
        "0x000c"
      ],
      "out_clusters": [
        "0x0004",
        "0x000c"
      ]
    },
    "242": {
      "profile_id": 41440,
      "device_type": "0x0061",
      "in_clusters": [],
      "out_clusters": [
        "0x0021"
      ]
    }
  },
  "manufacturer": "LUMI",
  "model": "lumi.plug.maeu01",
  "class": "zhaquirks.xiaomi.aqara.plug_mmeu01.Plug"
}
TB-SE commented 2 years ago

Hey sorry mate, i have updated my original post and answered these questions.

Cheers mate. Interesting. I'm just guessing now, so either the AU plugs hardware is not 100% equal to the EU version or they might use significantly different firmware.

Does anyone know if you can manually change the quirk?

TB-SE commented 2 years ago

The following issue seems to be linked/same as this one: https://github.com/home-assistant/core/issues/65292

martbock commented 2 years ago

I had the same problem with a freshly installed Home Assistant (running as a container on a RPi 4 with ZHA and a RaspBee II) and the SP-EUC01 Aqara smart plug.

For the maeu01 models: I am fairly certain I know what is wrong. is anyone willing to try to modify the quirk locally? if so, remove this line:

https://github.com/zigpy/zha-device-handlers/blob/b0bbbfb6b1f267b7d87d833b1ba5a5b0b127a9a1/zhaquirks/xiaomi/aqara/plug_mmeu01.py#L93 , then save the file and restart HA. After HA is back up, reconfigure the device via the reconfigure button on the device card.

z2m binds the clusters on these: https://github.com/Koenkk/zigbee-herdsman-converters/blob/66bfcdc9405abebe70906b279e96f12de73f26a7/devices/xiaomi.js#L1048

After applying this patch, most identified sensors are still showing "Not available" – however smartenergy_metering_summation_delivered now actually shows a value whereas before it was dead too.

What I find strange is the alleged device temperature value of 0,2°C which doesn't make any sense.

Screen Shot 2022-03-22 at 11 54 14

Stefano0042 commented 2 years ago

@dmulcahey ..summarizing an unwanted breaking change in 2022.m.r has pushed aqara plug into the "red water", adding lumi.plug.maeu01 in the quirks models list has not been enough to rescue the plug.

husjon commented 2 years ago

Hi, I purchased 3 of these EU plugs today hoping to monitor a few of my appliances.

I've got the following HA setup:

Version | core-2022.3.6
-- | --
Installation Type | Home Assistant Container
Development | false
Supervisor | false
Docker | true
User | root
Virtual Environment | false
Python Version | 3.9.9
Operating System Family | Linux
Operating System Version | 5.16.15-arch1-1
CPU Architecture | x86_64
Timezone | Europe/Oslo

Using SONOFF ZigBee 3.0

After going through all the comments I was able to get the plugs to report summed metering, but that's it. Following these steps.

  1. Comment out: https://github.com/zigpy/zha-device-handlers/blob/b0bbbfb6b1f267b7d87d833b1ba5a5b0b127a9a1/zhaquirks/xiaomi/aqara/plug_mmeu01.py#L93
  2. Remove existing device from the ZHA integration
  3. Add device

Before commenting out SKIP_CONFIGURATION: image

After commenting out SKIP_CONFIGURATION: image

Stefano0042 commented 2 years ago

@husjon Hi, but here we are still missing a fundamental sensor: active power, and I'm quite sure a multiplier factor x100 on device temperature.

husjon commented 2 years ago

@Stefano0042, yep. I was mainly adding my comment as a summary of what was mentioned previously. I'm still hoping for a solution as I mainly bought these plugs for appliance monitoring in addition to the switching.

I haven't been able to do much troubleshooting lately, but if someone could point me in the right direction I'll try to assist as needed.

jinovak commented 2 years ago

I have the same problem with newly added plugs. First one i bought and connect in January works well. Others were bought & connected in february and these are not working. All sensors are unavailable.

First plug connected in January (dont remember version of HA) is still working well Screenshot from 2022-03-25 09 37 03

Lately installed are unavailable - only on/off is available Screenshot from 2022-03-25 09 38 07

RossEm0174 commented 2 years ago

Hello, today I got my Aqara socket, and its currently also not working...

image
dmulcahey commented 2 years ago

Can someone with the MAEU01 plug please test the quirk in https://github.com/zigpy/zha-device-handlers/pull/1479

DO NOT USE THE CUSTOM DEPS ADDON

1 - Remove the existing device via the remove button on the device card in HA 2 - configure the quirk as a local quirk 3 - restart HA 4 - pair the plug to HA again

vinhui commented 2 years ago

I did that, but it didn't seem to make a difference for me.

The device after I re-paired it. image

The changes I made: image

FYI, currently running supervised version 2022.4.1 on my pi

torge70 commented 2 years ago

Can someone with the MAEU01 plug please test the quirk in #1479

DO NOT USE THE CUSTOM DEPS ADDON

1 - Remove the existing device via the remove button on the device card in HA 2 - configure the quirk as a local quirk 3 - restart HA 4 - pair the plug to HA again

Device is added. On_Off works. Power meter does not work.

husjon commented 2 years ago

On a clean 2022.4 docker instance I did the following steps:

  1. Patch local quirk
  2. Remove device
  3. Restart HA
  4. Add device

It is now missing sensor.electrical_measurement. (this was there with no quirk assigned) image

Also the device temperature is still off by a factor of 10. (same as no quirk) image

husjon commented 2 years ago

To add to my previous comment. Here is the same plug with no quirk assigned. (this includes the same steps as above, but with the quick assignment removed)

image

torge70 commented 2 years ago

Looks like this cluster value gives the current power consumtion in W. I don'd understand how to modify the quirk to pass this value correctly to HA.

ElectricalMeasurementCluster (Endpoint id: 1, Id: 0x0b04, Type: in) active_power (id: 0x050b) Value: 936

bendikrb commented 2 years ago

Same problem here. I have two (seemingly) identical plugs - one of which doesn't get quirk applied.. The quirk patch from @dmulcahey does not appear to make any difference here.. Following log output (with patch applied) which I suppose could reveal the/an issue (?);

2022-04-13 17:41:34 DEBUG (MainThread) [zigpy.application] Device is initialized <Device model='lumi.plug.maeu01' manuf='LUMI' nwk=0xEBCA ieee=54:ef:44:10:00:2c:7b:6d is_initialized=True>
2022-04-13 17:41:34 DEBUG (MainThread) [zigpy.quirks.registry] Checking quirks for LUMI lumi.plug.maeu01 (54:ef:44:10:00:2c:7b:6d)
2022-04-13 17:41:34 DEBUG (MainThread) [zigpy.quirks.registry] Considering <class 'zhaquirks.xiaomi.aqara.plug_mmeu01.Plug'>
2022-04-13 17:41:34 DEBUG (MainThread) [zigpy.quirks.registry] Fail because endpoint list mismatch: {1, 242} {1, 242, 21, 31}

Signature:

{
  "node_descriptor":"NodeDescriptor(logical_type=<LogicalType.Router: 1>, complex_descriptor_available=0, user_descriptor_available=0, reserved=0, aps_flags=0, frequency_band=<FrequencyBand.Freq2400MHz: 8>, mac_capability_flags=<MACCapabilityFlags.AllocateAddress|RxOnWhenIdle|MainsPowered|FullFunctionDevice: 142>, manufacturer_code=4447, maximum_buffer_size=127, maximum_incoming_transfer_size=100, server_mask=11264, maximum_outgoing_transfer_size=100, descriptor_capability_field=<DescriptorCapability.NONE: 0>, *allocate_address=True, *is_alternate_pan_coordinator=False, *is_coordinator=False, *is_end_device=False, *is_full_function_device=True, *is_mains_powered=True, *is_receiver_on_when_idle=True, *is_router=True, *is_security_capable=False)",
  "endpoints":{
    "1":{
      "profile_id":260,
      "device_type":"0x0051",
      "in_clusters":[
        "0x0000",
        "0x0002",
        "0x0003",
        "0x0004",
        "0x0005",
        "0x0006",
        "0xfcc0"
      ],
      "out_clusters":[
        "0x000a",
        "0x0019"
      ]
    },
    "21":{
      "profile_id":260,
      "device_type":"0x0051",
      "in_clusters":[
        "0x000c"
      ],
      "out_clusters":[

      ]
    },
    "31":{
      "profile_id":260,
      "device_type":"0x0051",
      "in_clusters":[
        "0x000c"
      ],
      "out_clusters":[

      ]
    },
    "242":{
      "profile_id":41440,
      "device_type":"0x0061",
      "in_clusters":[

      ],
      "out_clusters":[
        "0x0021"
      ]
    }
  }
}
cailliaudBlablacar commented 2 years ago

Hello @dmulcahey

When do you plan to merge your nice fix :D ? https://github.com/zigpy/zha-device-handlers/pull/1479 I just bought 4 lumi.plug.maeu01 and i would like to be able to have the metrics on consumption

Thank you again for your work

husjon commented 2 years ago

Hi @dmulcahey

I've set up the environment locally and been looking at your PR and the only way to get it to work is to add .cluster_id to the missing input_clusters. image

This makes it so that both electrical_measurement and smartenergy_metering_summation_delivered correctly shows their values. The plug state and measurements are also updated in realtime when the button is pressed on the physical unit. The device_temperature is still off by a factory of 100, but is not that big of a deal currently.

Currently this is what I've got going. image

dmulcahey commented 2 years ago

What clusters did you change?

husjon commented 2 years ago

Here are the differences between your PR: https://github.com/husjon/zha-device-handlers/commit/ddae1020e581822a58bae1621fa4494bbd690543

dmulcahey commented 2 years ago

OK that will break the other plug… give me a few to push a change to the PR and then can you test it again if you have a few?

dmulcahey commented 2 years ago

Here are the differences between your PR: husjon@ddae102

So the more I think about this.. can you try something for me? Just remove the quirk class PlugMAEU01 completely. Remove the plug from HA / ZHA, wait about 30s then restart HA and re-pair it please. If it works we can nix the quirk until someone wants to look into DeviceTemperature

dmulcahey commented 2 years ago

I pushed this change to the branch

husjon commented 2 years ago

Sure

husjon commented 2 years ago

This is how the plug is configured without the quirk class. Same as with the cluster_id assignment.

image

dmulcahey commented 2 years ago

So aside from device temperature it is all working?

husjon commented 2 years ago

Yeah, except for the device temperature everything seem to work as expected. The plug on_off and electrical_measurement is updated in realtime when the physical button on the is pressed.

Also as for smartenergy_metering_summation_delivered it seem to be correct. I need to run my current load for a while to get a more valid summary.

dmulcahey commented 2 years ago

OK if this tracks true we can just remove it for now. Thanks for testing.

husjon commented 2 years ago

Summation is working as expected.

Happy to help.

dmulcahey commented 2 years ago

Anyone else on this thread want to give this a shot?

tetele commented 2 years ago

Anyone else on this thread want to give this a shot?

I did. It took about 5-6 tries to re-pair the plug correctly (with all endpoints, clusters, resulting entities etc.). However, once paired, it reports a 0 power draw, although that's not true.

2022-04-21_18-29-58

tetele commented 2 years ago

Btw, I'm not sure if it's really important, but I haven't used a custom quirk. I modified the quirk directly in the Docker container on HAOS. Is that common practice?

dmulcahey commented 2 years ago

Btw, I'm not sure if it's really important, but I haven't used a custom quirk. I modified the quirk directly in the Docker container on HAOS. Is that common practice?

We do not support doing this. But it does work. Can you remove it again please and provide the debug logs from pairing it again?

tetele commented 2 years ago

We do not support doing this. But it does work.

What's the recommended way, then? Simply create a copy of the quirk in /config/custom_zha_quirks?

Can you remove it again please and provide the debug logs from pairing it again?

Sure! Here you go!

dmulcahey commented 2 years ago

We do not support doing this. But it does work.

What's the recommended way, then? Simply create a copy of the quirk in /config/custom_zha_quirks?

Can you remove it again please and provide the debug logs from pairing it again?

Sure! Here you go!

Oof that thing tried to configure several times and there are various failures. Let things settle down then go to the device card in HA and click the reconfigure button and run through that dialog please.

tetele commented 2 years ago

Is this enough? It didn't do anything differently after reconfiguration.

2022-04-21_18-58-10

dmulcahey commented 2 years ago

Well that says it all worked… but you are saying the plug still isn’t working correctly in HA?

tetele commented 2 years ago

I am saying that when I turn on a ~1kW load in the plug, it reports a power drain of 0.0W, as per HA. I've also tried reading directly from the cluster (ElectricalMeasurement 0x0b04 > active_power 0x050b) and the value reported is also 0.

Same behavior with 2 identical plugs with different loads.

husjon commented 2 years ago

I see the same reconfiguration parameters that @tetele is seeing, but on my end all 3 of my plugs are all working as expected. :thinking:

Below you'll find the logs from my pairing attempts (isolated to one of the plugs) and also how the environment was set up (start_ha.sh) at the bottom of the gist. https://gist.github.com/husjon/16b4eaeeaf3a3019f861cdc07e6215ae

Hope this helps.

Stefano0042 commented 2 years ago

On core-2022.4.3 HA Core installation:

Patch local quirk Remove device Restart HA Add device

Quirk is skipped all the time...., so nothing different, clear.

Inervo commented 2 years ago

I'm having the same issue as @tetele: my 2 plugs are recognized, but without any value for electrical_measurement or summation_delivered.

Tried on core-2022.4.6. Patch local quirk Remove device Restart HA Add device

balves42 commented 2 years ago

I can confirm the same results as @husjon . I'm currently charging my phone and the values seem to be ok (except temperature). I'm using a custom quirk with the file downloaded from the fix-xiaomi-maeu01 branch

Stefano0042 commented 2 years ago

@husjon ...your app_version (0x0001) is?

husjon commented 2 years ago

@Stefano0042, my test instance was using docker:

homeassistant/home-assistant   2022.4.6     c75bec8a4f57   27 hours ago    1.25GB
homeassistant/home-assistant   stable       c75bec8a4f57   27 hours ago    1.25GB

If that was what you ment. :)

tetele commented 2 years ago

I think he meant the firmware version, which you get by going to the device page > Manage clusters > Cluster: Basic (0x0000) > Attribute: app_version (0x0001) > Get Zigbee attribute

Screenshot_20220422-073453_Home Assistant

Stefano0042 commented 2 years ago

exactly Tudor, thank you, after the last available upgrade is 32 as all my maeu01 are today.

Stefano0042 commented 2 years ago

..maybe I'm wrong but dut (device under test) firmware alignment should be part of the preconditions. coordinator type and ver..., ...

torge70 commented 2 years ago

I have to different firmware revisions on my plugs. The old one with firmware 22 was added before xmas and report power and consumption. The new ones with firmware 32 does not report power and consumption. On-Off works on both.

I have not tried to reconfigure or add the old one after installing the latest custom quirk.

Stefano0042 commented 2 years ago

..this is in line with my "32" case, no power and consumption, the only recognizable quirk was the old one with maeu01 in the model list, at the root of proliferations of "unknown" entities. Two firmware, two different behaviors.

husjon commented 2 years ago

Thanks @tetele, all 3 of my plugs shows app_version: 32

image