dresden-elektronik / deconz-rest-plugin

deCONZ REST-API plugin to control ZigBee devices
BSD 3-Clause "New" or "Revised" License
1.9k stars 503 forks source link

Hue Dimmer Switch (2021 model) #4287

Closed dieugab closed 3 years ago

dieugab commented 3 years ago

Device

Screenshots

Basic cluster Node info Power cluster

ebaauw commented 3 years ago

I'll have a look this weekend. Should be fairly similar to the Hue smart button, with the two endpoints of the older model combined into one endpoint.

ebaauw commented 3 years ago

Ha, looking at the code, the ZHASwitch resource is created, and then removed. There's some archaic code to fix the original Hue dimmer switch being exposed wrongly (in a version from 2017 or older), forcing the endpoint to 0x02. https://github.com/dresden-elektronik/deconz-rest-plugin/blob/f46e3321715f755a057c19992e51ce9cf6f01e35/de_web_plugin.cpp#L5649-L5657

@manup I propose to remove this block altogether, would you agree? Alternatively, I could add a check for RWL022.

EDIT Bad idea, the setting of the endpoint is quite crucial, or the plugin will continue to create new, identical resources, see below.

Note to self: don't try and support devices you don't have your hands on. The code is too complex to understand without extensive testing...

ebaauw commented 3 years ago

Couldn't resist and got me one. And a good thing I did, too, because it doesn't work. On pairing, two identical resources are created, both for non-existing endpoint 0x02:

{
  "14": {
    "config": {
      "battery": null,
      "group": "18",
      "on": true,
      "reachable": true
    },
    "ep": 2,
    "etag": "9c2ef3035c5520e72ee1c8751a98a058",
    "lastseen": "2021-02-05T18:03Z",
    "manufacturername": "Philips",
    "mode": 1,
    "modelid": "RWL022",
    "name": "RWL022 14",
    "state": {
      "buttonevent": null,
      "eventduration": null,
      "lastupdated": "none"
    },
    "swversion": "2.42.0_h1F5E860",
    "type": "ZHASwitch",
    "uniqueid": "00:17:88:01:09:27:59:e1-02-fc00"
  },
  "15": {
    "config": {
      "battery": null,
      "group": "19",
      "on": true,
      "reachable": true
    },
    "ep": 2,
    "etag": "d0eebba8e619781eb9119c1862599b21",
    "lastseen": "2021-02-05T18:03Z",
    "manufacturername": "Philips",
    "mode": 1,
    "modelid": "RWL022",
    "name": "Switch 15",
    "state": {
      "buttonevent": null,
      "eventduration": null,
      "lastupdated": "none"
    },
    "swversion": "2.42.0_h1F5E860",
    "type": "ZHASwitch",
    "uniqueid": "00:17:88:01:09:27:59:e1-02-fc00"
  }
}

After patching the database, deleting the second resource and correcting uniqueid and the fingerprint to reflect endpoint 0x01, the sensor works as expected. It's just the pairing that goes wrong brilliantly.

{
  "config": {
    "battery": 100,
    "group": "18",
    "on": true,
    "reachable": true
  },
  "ep": 1,
  "etag": "282e7d4a6ba93d0670b3d2e8723ebf36",
  "lastseen": "2021-02-05T18:12Z",
  "manufacturername": "Signify Netherlands B.V.",
  "mode": 1,
  "modelid": "RWL022",
  "name": "Hue dimmer",
  "state": {
    "buttonevent": 4002,
    "eventduration": 1,
    "lastupdated": "2021-02-05T18:11:51.740"
  },
  "swversion": "2.42.0_h1F5E860",
  "type": "ZHASwitch",
  "uniqueid": "00:17:88:01:09:27:59:e1-02-fc00"
}

Note that the firmware reports Manufacturer Name "Signify Netherlands B.V.". When reading the Basic cluster from the GUI, the manufacturername attribute is changed to that value. I think its time to report the new name for all old Hue devices, as the Hue bridge has been doing since apiversion 1.36.0.

ebaauw commented 3 years ago

Found another place in the code where endpoint was forced to 0x02 (there's always one more place to whitelist...). After changing that, the new model pairs OK.

pimw1 commented 3 years ago

Hi, the new implementation is not working for me. I'm using version 2.9.3. See the topic below for more details: https://community.home-assistant.io/t/philips-hue-switch-new-generation/281525

ebaauw commented 3 years ago

"Not working" is not really informative. Please specify:

Note that the API might expose manufacturername as "Philips" at first, and as `"Signify Netherlands B.V." after re-reading the Basic cluster attributes.

Any issues with Phoscon or with Home Assistant should be addressed in their respective repositories - nothing we can do about that here.

Mimiix commented 3 years ago

@pimw1 This is not how it works. Please open a new issue or provide sufficient details here.

pimw1 commented 3 years ago

Hi guys,

I gave my Conbee II a hard reset, as well as the phoscon/deconz applications. I've added my 10 lights again, as well as 2 smart plugs. The 5 motion sensors and door sensor will be done later, as well as the Hue Gen1 switches. For now, this setup allows me to focus on the gen 2 Hue switch.

I'll try to answer your questions as good as possible:

Yes, it does: "RWL022 8". It seems not connected to my Conbee II stick. image

pimw1 commented 3 years ago

Hi all, i've added all my devices again and restructured the overview. Hopefully this helps: image

The two devices at the very bottom are a mystery for me. After the above post, i've removed the battery from the RWL22, removed the device from the Conbee stick, and it still pops up again in the overview. It sounds to me like some kind of database is internally messed up a bit. How else could it popup again after removing the battery+removal from the Conbee removal? The 0xE65C5 is a complete mystery for me as well.

dieugab commented 3 years ago

FYI it's been working for me on the latest stable. I've got 3 2021 dimmers setup and working. They don't show up in the Phoscon UI (as expected), but are visible through the VNC. Button mapping works perfectly through Home Assistant

pimw1 commented 3 years ago

Thanks! Could you please share which steps you took to connect the devices?

jvalousek commented 3 years ago

Hey, first of all, thank you for your work, it's excellent! It looks (probably) fine for me in cluster view. Button value is changed if dimmer's button pressed (1,2,3,4). I have 2 new controllers, both are connected but can not see them in Phoscon. From Home assistant view, there is just entities with battery status (deconz_event is not triggered either). I'm happy to help with some additional screenshots or specifications, but I don't know where or what to look exactly in the DeConz environment.

Snímek obrazovky 2021-02-18 v 13 38 54

pimw1 commented 3 years ago

Hi all,

I've removed the RWL022 again in Deconz, and placed the battery back in the switch after that. The device is now succesfully connected in Deconz, and it is also working in Home Assistant (battery status + all 4 buttons). In Phoscon, it not showing up.

image

So, all in all, it is working now, and i have absolutely no clue what i did differently as compared to yesterday. It looks kinda 'random'/hit-and-miss.

jvalousek commented 3 years ago

@pimw1 you can see buttons as entities in Home assistant? I tried to repeat your procedure (remove battery reset device and add it once again to DeConz), but there are still just battery statuses in HA.

Snímek obrazovky 2021-02-18 v 15 11 34

There are even some errors in logs in DeConz integration:

15:16:37:701 [INFO] - No button map for: RWL022, unicast to: 0x0000, endpoint: 0x01, cluster: 0xFC00, command: 0x00, payload: 0100003000210000, zclSeq: 100 15:16:37:823 [INFO] - No button map for: RWL022, unicast to: 0x0000, endpoint: 0x01, cluster: 0xFC00, command: 0x00, payload: 0100003002210100, zclSeq: 101 15:16:38:274 [INFO] - No button map for: RWL022, unicast to: 0x0000, endpoint: 0x01, cluster: 0xFC00, command: 0x00, payload: 0200003000210000, zclSeq: 103 15:16:38:395 [INFO] - No button map for: RWL022, unicast to: 0x0000, endpoint: 0x01, cluster: 0xFC00, command: 0x00, payload: 0200003002210100, zclSeq: 105 15:16:39:074 [INFO] - No button map for: RWL022, unicast to: 0x0000, endpoint: 0x01, cluster: 0xFC00, command: 0x00, payload: 0300003000210000, zclSeq: 106 15:16:39:196 [INFO] - No button map for: RWL022, unicast to: 0x0000, endpoint: 0x01, cluster: 0xFC00, command: 0x00, payload: 0300003002210100, zclSeq: 108 15:16:39:779 [INFO] - No button map for: RWL022, unicast to: 0x0000, endpoint: 0x01, cluster: 0xFC00, command: 0x00, payload: 0400003000210000, zclSeq: 109 15:16:39:954 [INFO] - No button map for: RWL022, unicast to: 0x0000, endpoint: 0x01, cluster: 0xFC00, command: 0x00, payload: 0400003002210100, zclSeq: 110

dieugab commented 3 years ago

@jvalousek you can’t see buttons as entities with the deCONZ integration in HA. You need to go to Developer tools → Events → Listen to events, input “deconz_event” and start listening.

press a button and information should show up. You can use that to create automations.

jvalousek commented 3 years ago

@gvssr thank you, i'm familiar with this, but as pointed out before, ”deconz_event” is not triggered by any of button either.

pimw1 commented 3 years ago

That is working for me (see screenshot below). Yesterday, this was not working. I am unable to figur out why the same procedures are working today, but not yesterday. image

jvalousek commented 3 years ago

Got it! Because i tried add devices before it has been supported by deconz-rest-plugin, there was an missconfiguration in core database. In this case, device has to be fully removed (not just by "delete node" option in DeConz VNC). Only direct request to API worked for me:

curl -X DELETE \
  http://1.1.1.101:40850/api/APITOKEN/sensors/29 \
  -d '{"reset": true}'

Now it work's like a charm :)

Thank you!

pimw1 commented 3 years ago

I think this might have been the root cause for me as well! Yesterday i completely removed Deconz, Phoscon, and i fully reset the Conbee II. I spend the whole day reconfiguring everything. This morning, i put the battery of the Gen 2 switch back in.

The similarity between our cases is: (1) we both added the device first with old Conbee software, (2) we both removed the device drastically afterwards (you did it more elegantly than i did). I think we both suffered from an misconfiguration in the database.

If this theory is right, we might have uncovered a very nasty bug: unsupported devices creating database issues.

jvalousek commented 3 years ago

Yes, it looks definitely like same issue for both of us. Actually there is an open feature request about it. It's just not well documented how to properly remove device from DeConz.

See #2064

pimw1 commented 3 years ago

Thanks! Another part that seems troublesome to me is that, while connecting in Phoscon, sometimes (1 out of 10) it will be able to connect. However, the result of that was two-fold:

All in all, it seems that some serious bugs are awaiting to be fixed ;)

ebaauw commented 3 years ago

I think there's a bug in the current implementation: when the switch is restored from the database (i.e. when you restart deCONZ, after successfully pairing the Hue dimmer switch (2021)), the wrong (non-existent) endpoint 2 is configured. uniqueid now ends in -02-fc00 (instead of -01-fc00).

KennethLavrsen commented 3 years ago

I also bought one of these (did not know there was a new one so unhappily surprised when I took it out of the package)

I try to pair it. It never shows up in Phoscon.

But as I can read from this thread it sends rest events.

But how do you rename it when it does not show up in Phoscon? I saw this bug report getting closed. How we get it added to Phoscon?

KennethLavrsen commented 3 years ago

It got even worse. I tried to see if there was a beta upgrade of Deconz and there was an upgrade

After that I had 7 Dimmer Switches. I deleted them all but one. And then tried to rename it to the name I want. The minute I do that all light in the house flashes like a switch has been discovered and it renames itself back to default.

Then I deleted it and tried to readd it again. It never adds again but exists in the rest API as rwl022_104 and no way to rename it in Phoscon.

I do not understand what is going on.

KennethLavrsen commented 3 years ago

Just follow up. I ended up with the whole installation totally toasted. The database got screwed up to a point of no return. I finally had to restore an old backup (2 weeks old) and then had to re-add many of my old Philips dimmers again because they had been reset in my attempt to fix the problems.

I will say this. If you want to try to add a RWL022 to your deCONZ, make a fresh backup and prepare yourself for trouble. And the trouble will occur when you restart deconz.

or better wait for @ebaauw to come back here and tell us the issue has been fixed in a beta.

ebaauw commented 3 years ago

But how do you rename it when it does not show up in Phoscon?

Use the REST API.

I saw this bug report getting closed.

"This" is an issue requesting device support, which was closed when the original PR that links to this issue was merged. I re-opened it, when I noticed the wrong uniqueid after database restore. Not sure if that's related to multiple resources being created - I need to see these resources. I did notice some junk in my database - but this is a dev/test installation so could also have been created while developing.

How we get it added to Phoscon?

Open a device support request in the Phoson repo, https://github.com/dresden-elektronik/phoscon-app-beta/issues. Best link to this issue. Note that Phoscon is not open source, so no-one here can help with that. See deCONZ for Dummies in the Wiki for details.

And the trouble will occur when you restart deconz.

That would probably be related to https://github.com/dresden-elektronik/deconz-rest-plugin/issues/4287#issuecomment-797550925. I actually fixed that in #4541, but failed to mention it. Sorry for that. The PR is included v2.10.4, so the 2021 Hue dimmer should now also work after restarting deCONZ.

github-actions[bot] commented 3 years ago

As there has not been any response in 21 days, this issue has been automatically marked as stale. At OP: Please either close this issue or keep it active It will be closed in 7 days if no further activity occurs.

KennethLavrsen commented 3 years ago

I had so much trouble when I added the new hue switch the first time. This bug caused my entire database to be corrupted and clicking any old hue remote made all lights in the house go on and off. I had to restore deconz from old backup to get things to work. So I am reluctant at trying again.

There has been a couple of releases since this. Has this issue been addressed? Or does Deconz no longer support Philips Hue devices?

pimw1 commented 3 years ago

I had similar issues, and had to completely reset my Conbee II to the factory defaults. It took me a full weekend to restore everything. The bug turns out to be very, very, very nasty. It is all running now, but i don't dare to connect any new Hue switches until this issue has been fixed. I'm not sure about the priority of this issue, but this is more severe than just "device x is not supported": it can bring everything down.

Mimiix commented 3 years ago

@ebaauw Can you comment on this issues?

ebaauw commented 3 years ago

No more than I have before.

KennethLavrsen commented 3 years ago

@ebaauw I am not sure I understood your statement.

Do you believe this issue was fixed with #4541? The thing that makes me doubt is that you kept this bug report open.

ebaauw commented 3 years ago

I honesty don't know, as I've mentioned before, I would need to see the resources. Happy to close this issue, if it makes you feel better.

I've added the 2021 Hue dimmer to my production network under v2.10.4 and haven't seen any weirdness in the last three weeks.

KennethLavrsen commented 3 years ago

OK. I will take a backup and give it a try this weekend and report back

ebaauw commented 3 years ago

In case it goes wrong, please save the output of a GET of /api/username/ before restoring the database.

KennethLavrsen commented 3 years ago

I tried to add it again.

It went better.

When I enable the joining in Phoscon you can see on the device it gets added but Phoscon shows nothing. Nothing is added.

When you look at the Deconz GUI app you see the device. In my installation it is named RWL022 109.

In Home Assistant I see events from device RWL022 109.

There is no human user interface that let me rename that node. Since it is invisible in Phoscon you cannot rename it there.

In the Deconz GUI the feature to change the name of the device is still broken like it has been forever. You can edit the text field with the name and it says it send the name change but nothing ever happens no matter how many times you try and how many times you try and click on the device.

Maybe I can rename it via the Rest API? I have not tried yet. I will. I ran out of time. It is going to take time to figure out how to do that. But that is not a proper user interface. It is only the most hard headed geeks that can manage that.

The database does not get screwed up at least so if the name can be changed via API then it seems the hard headed geeks can use it now.

So overall situation is - there is progress.

But Dresden Elektronik cannot claim this device is supported when the Phoscon app still will not recognise it. And it is THE MAIN BRAND device - Philips Hue we are talking about here. It is one of the main selling features for the Deconz/Phoscon product. I know you volunteer rest API developers cannot do anything about this. This message is for Dresden Elektronik.

KennethLavrsen commented 3 years ago

In case it is of any help. Here is the JSON part of the rest response (beautified) for the device.

    "109": {
      "config": {
        "battery": 100,
        "group": "30",
        "on": true,
        "reachable": true
      },
      "ep": 1,
      "etag": "714d186f062ebe8b682b2716aa6b6540",
      "lastseen": "2021-04-17T12:28Z",
      "manufacturername": "Signify Netherlands B.V.",
      "mode": 1,
      "modelid": "RWL022",
      "name": "RWL022 109",
      "state": {
        "buttonevent": 4002,
        "eventduration": 1,
        "lastupdated": "2021-04-17T12:03:50.418"
      },
      "swversion": "2.42.0_h1F5E860",
      "type": "ZHASwitch",
      "uniqueid": "00:17:88:01:09:25:f1:6b-01-fc00"
    }
ebaauw commented 3 years ago

it gets added but Phoscon shows nothing.

It needs to be supported by Phoscon. As that's closed source, I cannot help there.

In the Deconz GUI the feature to change the name of the device is still broken like it has been forever.

That feature changes the User descriptor of the device, but only a few devices actually support this. Even when they do, it changes the label on the node in the GUI, not the resource's name attribute.

It is going to take time to figure out how to do that.

Do a PUT to /api/username/sensors/109 with a body of {"name": "My Favourite New Name"}. I prefer to use ph (which is bundled with Homebridge Hue), but any REST client will do, e.g. Postman or even curl.

This message is for Dresden Elektronik.

Best contact their support, then.

Here is the JSON part of the rest response (beautified) for the device.

That's looking good. Did you already restart deCONZ?

KennethLavrsen commented 3 years ago

Hi.

First thanks for your reply. I had found the API way and I could change the name. I had to restart Home Assistant before it started accepting the new name in the events. Just so the HA users reading this knows.

Yes. I have restarted deCONZ a couple of times. And no corrupted database.

Doesn't some of you developers work for Dresden Elektronik? That was my impression. And I cannot believe Dresden Elektronik and the people coding the closed source part of the product do not follow their own github.

ebaauw commented 3 years ago

Some of the dresden developers are active here, most notably Manu. I don't think they read all the GitHub issues in detail, see #3765.

I think we can close this issue, then?

KennethLavrsen commented 3 years ago

I just realize that the Phoscon beta github has been activated (used to be a dead locked place last I checked with may be a year ago)

https://github.com/dresden-elektronik/phoscon-app-beta/issues/367 covers the remaining open issue which means this one can be closed.

pimw1 commented 3 years ago

Hi Erik, after updating deconz, i've bought two new hue gen2 switches. They are showing up correctly in Deconz and working as expected in Home Assistant. Thank you very much for your help and effort!

eakoning commented 3 years ago

Hello @pimw1 , could you perhaps explain to me how you're using this device from inside Home Assistant? I'm running the Recommended installation method of that and thereby using Deconz and Phoscon, but I don't see it in the Phoscon Web App. Only the only old Dimmer switch and tap are there. How would I go about to add it to HA and actually control some lights with it?

pimw1 commented 3 years ago

Hi @eakoning . It's not implemented in Phoscon yet, but it is implemented in Deconz. You can follow the Phoscon issue on github: https://github.com/dresden-elektronik/phoscon-app-beta/issues/367

The steps are as follows:

MiAutomations commented 2 years ago

Hello to all

I'm not able to triger a event in the atomations using the folowing code

''' event_type: deconz_event data: id: wall_switch_module unique_id: 00:17:XX:XX:XX:XX:35:24 event: 1002 device_id: a9254b97efXX50XX1xXe8779ba32f2a5 origin: LOCAL time_fired: '2022-04-16T14:35:19.342491+00:00' context: id: cd12201d75e41a103XXXXX502ce801216 parent_id: null user_id: null ''' I successfully "listening" the event deconz_event , but I cant use in the automations ...

image

Can you please help me in oder to solve this issue ?

Thank you in advance for your help

pimw1 commented 2 years ago

Hi, this is not a question for Deconz but for Home Assistant. Best is to seek help on the Home Assistant forum or the discord channel. Since i am a home assistant user, i'll help you a bit by showing you an example from my automation trigger. For further question, kindly go to those other channels.

alias: Woonkamer licht handmatig
description: ''
trigger:
  - device_id: xxxxxxxx
    domain: deconz
    platform: device
    type: remote_button_short_press
    subtype: turn_on
    id: rechterkant aan/uit
MiAutomations commented 2 years ago

alias: Woonkamer licht handmatig description: '' trigger:

  • device_id: xxxxxxxx domain: deconz platform: device type: remote_button_short_press subtype: turn_on id: rechterkant aan/uit

Thank you for you reply and support !

But I think this is related ... Sorry for this ... I still facing this issue and I think that is related to some of the "commands or parameters" once I have others events working as a triggers ...

Mimiix commented 2 years ago

@MiAutomations it is not related to deconz but to the implementation of deconz by a third party.

Feel free to ask on discord or forums.