PTM 215Z enOcean slow respons randomly

samuele2723 commented 5 months ago

What happened?

Hello, i write here to have some support for my home switches because my wife is crazu with me!!

I have a large setup with Sonoff CC2531 coordinator, my network is composed by 190 devices across 4 levels (garage, ground, 1st, 2nd). Overall the network works good, means i don't see delays of any kind when i use from home assistant app or from Z2M interface the lights.

Only random delays i have, are when using my switches around the house, that are all PTM 215Z enOcean. And my wife is mainly using these physical buttons to control the lights.

[https://zigbee2mqtt.io/devices/PTM_215Z.html#enocean-ptm%2520215z]

Sometimes, when i click, the light is switch on\off instantly, (i could not even say there is a delay). Some other times it takes 1 to 3 seconds. This happens also in the same switch in the same moment, means if i keep push on\off the delays changes time by time.

I can't say it's an hardware issue, because i have 25 of them and all are affected (especially ones far from coordinator that's at garage level) and also because i've used these with Philips Hue Bridge and they seemed to be more responsive in that setup

What did you expect to happen?

I expect that there is no delay or minimum and repetitible always similar when i use switches. They should be almost instant in their responsiveness, I don't also want to see difference from ground floor or second floor, i would like to understand how to prioritize message responsiveness from these switches

How to reproduce it (minimal and precise)

I can reproduce it every time, and is not related to what light i turn on, or how (home assistant or MQTT) seems related to the delay of the button transmission of PTM 215Z enOcean itself.

It can also be noticed by watching the state change of click actions of PTM 215Z enOcean in the dashboard, you can perceive the message is received in delay from the switch.

Zigbee2MQTT version

1.38.0

Adapter firmware version

20210708

Adapter

ZDONGLE P 2652 sonoff zStack3x0

Setup

Home assistant OS with MQTT and Z2m Addon

Debug log

No response

chris-1243 commented 4 months ago

@burmistrzak I just remember I have a RF-Star dongle laying somewhere for a backup of my main network😁.

Let's use it and share the result on github or via the link provided. It will be "safer" like this

chris-1243 commented 4 months ago

Finally, I created a test network and you get below two links with the successful and unsuccessful pairing. I did not have more time to try to set a command-based device model via NFC. I need to dig a bit more.

I hope it may help to find out what is going on with this model.

burmistrzak commented 4 months ago

Finally, I created a test network and you get below two links with the successful and unsuccessful pairing.

@chris-1243 Great! I have the links open and 'am waiting for you to start the transfer. ✌️

I did not have more time to try to set a command-based device model via NFC. I need to dig a bit more.

Please share your experience! Quite interested in how well the NFC thing actually works.

chris-1243 commented 4 months ago

The links again...my computer turned off and everything has been closed......

failed

successful

my apologies....

burmistrzak commented 4 months ago

@chris-1243 No worries! 😊 Already got the successful pcap, however the failed one is still connecting... Maybe try again? Also, I unfortunately need the network key to decrypt the packets from the GP Proxy to the coordinator. Can you put it in a .txt file and send it the same way?

chris-1243 commented 4 months ago

Sure. May I provide you the key in hexadecimal ? The one used in wireshark or do you need the standard numbers ?

All transfers are ok from my side

burmistrzak commented 4 months ago

Sure. May I provide you the key in hexadecimal ? The one used in wireshark or do you need the standard numbers ?

Yup, the one for Wireshark.

All transfers are ok from my side

Hmm... Can you package both pcap files, as well as the network key (as a .txt), in a single zip and upload that? Less of a chance something gets lost. 😅

chris-1243 commented 4 months ago

Here we go

PTM216Z

burmistrzak commented 4 months ago

@chris-1243 Did you keep the tab with the transfer open? Still Connecting to Sender on my end... Give it another try.

chris-1243 commented 4 months ago

@burmistrzak yes I did. For me, the transfer is completed.

A new link?

burmistrzak commented 4 months ago

@chris-1243 Make sure the link is not clickable. Otherwise GitHub (?) might try to fetch a link preview or something. Best to remove the https:// so it's just regular text.

chris-1243 commented 4 months ago

onetimedrop.app/#DxOMFwTK4sKtiZ1VtxklKf9PrqtKZ32U7rcu64NhKQiFzBJKCL5elyQSFVERwW6R2oeqC0LJGippwFg7Upz08Y1

Let's try this

burmistrzak commented 4 months ago

@chris-1243 Got it! 🙌 Unfortunately... The most interesting part i.e. GP Commissioning Notification from the failed attempt is missing. You have sanitized the packet capture just a little bit too good. 😉

chris-1243 commented 4 months ago

Good I am..... I will make an other try tomorrow. The light used is in a kid room and it is not the the best moment to turn on/off a light.

I just had a look how to configure via NFC. It seems straight forward. At least, the Enocean app is easy to use. The Z2M documentation needs to be update as it doesn't point to the latest user manual.

burmistrzak commented 4 months ago

@chris-1243 No problem. How about you reset/configure the PTM216Z tomorrow, make another attempt at pairing with 9290022166, and capture some fresh packets?

If it works, perfect! If not, we'll have something interesting to look at. 😉

chris-1243 commented 4 months ago

Please share your experience! Quite interested in how well the NFC thing actually works.

I used an IOS smartphone and NFC works nicely. I could change the channel, commisioning mode and send the learn telegram. Unfortunately, I could not set any other configuration. The app did not provide other parameters to be set. It is very useful when the device is in a single rocker

burmistrzak commented 4 months ago

I used an IOS smartphone and NFC works nicely. I could change the channel, commisioning mode and send the learn telegram. Unfortunately, I could not set any other configuration. The app did not provide other parameters to be set. It is very useful when the device is in a single rocker

Well, that's more than enough for me! 😉 Quite impressive what EnOcean is able to accomplish with such little energy. Really like these things, even though they're a bit louder than regular switches in some cases. I'll have to someday look into acoustic dampening, to perfect them.

chris-1243 commented 4 months ago

Quite impressive what EnOcean is able to accomplish with such little energy.

Yes and I do use NFC for pairing now (PTM216Z). No need anymore to click more than 7s on a button and to confirm the channel. Really great option.

Really like these things, even though they're a bit louder than regular switches in some cases. I'll have to someday look into acoustic dampening, to perfect them.

PTM216Z seems a bit louder than PTM215Z for me... and I don't like the click feeling while using the PTM216Z. As I have only one and for testing purpose only. I'm fine. They might be a bit louder, especially during the night when it is really quiet.

Below, my try of the day. I disabled all the filter in Wireshark and I did multiple tries. One interesting point, I did received errors in Z2M.

zigbee2mqtt  | [2024-06-14 13:12:14] error:     zh:controller:greenpower: Error: Data request failed with error: 'No network route' (205)

The link:

onetimedrop.app/#MnGceqV1g5h5m2fI6EaN45VCujugdGSot8p1XBaRefy50SRjGMPvhrZTF4PahKbsgKOc1dwaMSHxw2C8CMhiU71

burmistrzak commented 4 months ago

@chris-1243 Oh, the new EnOcean switches feel different? Are they hard to press? Can you describe a bit?

Also can you please create a new link? 🙈

chris-1243 commented 4 months ago

onetimedrop.app/#ixsNdIwBk2Yee6VFSTisl2UaSNUHYMnvAF83kNRzS4irWepa9G3DikNYHSi1EHFBK7qzq3hKt0wOnCysa7Qh411

I guess it is just a matter of feeling. PTM216Z seems a bit harder to press (it is new compared to my PTM215Z). It is more the sounds which, in my opinion, is more present while clicking.

What I don't like compared to the PTM215Z is how the commands are sent, especially the release. It is always the same release command sent. While using the PTM215Z/ZE, you normally have press_x then release_x. PTM216Z, it is press_x then just release. More complexity when you create automation in HA or any other tool.

burmistrzak commented 4 months ago

@chris-1243 Eh, still connecting on my end. Whatever, just upload the latest *pcapng as an attachment here. Thankfully, I already have the network key received securely. These encrypted packets are pretty much the same as what's already in the air around your home. 😉

I guess it is just a matter of feeling. PTM216Z seems a bit harder to press (it is new compared to my PTM215Z). It is more the sounds which, in my opinion, is more present while clicking.

Are your EnOcean switches replacing traditional light switches, i.e. is there a hole in wall behind? I'll definitely have to experiment with different types of fire-rated acoustic foam. When these switches are mounted on solid concrete or brick, they're surprisingly quiet in comparison.

What I don't like compared to the PTM215Z is how the commands are sent, especially the release. It is always the same release command sent. While using the PTM215Z/ZE, you normally have press_x then release_x. PTM216Z, it is press_x then just release. More complexity when you create automation in HA or any other tool.

Can't this be changed by switching to a command-based mode?

chris-1243 commented 4 months ago

Pairing_PTM216Z_failed_2024.06.14.zip It is a test network....

Are your EnOcean switches replacing traditional light switches, i.e. is there a hole in wall behind? I'll definitely have to experiment with different types of fire-rated acoustic foam. When these switches are mounted on solid concrete or brick, they're surprisingly quiet in comparison.

Both way. On solid wall or to replace traditional light switches. It may depend on your wall maybe and the acoustic foam or material you have behind. For me, the click sound is almost similar.

Can't this be changed by switching to a command-based mode?

I need to read again the user manual. The description above reflects the generic command-based mode. Let's have a closer look.

burmistrzak commented 4 months ago

@chris-1243 AFAICT, the coordinator (Z2M) should have responded to packet No. 178 with GP Pairing... Do you have debug logs from Z2M for these failed attempts?

chris-1243 commented 4 months ago

Now, I have one...

log.txt

burmistrzak commented 4 months ago

@chris-1243 Got a packet capture as well? 😅

chris-1243 commented 4 months ago

In fact...not really😱🫣

I don't have that much spare time now. I will provide you a full debug log and a packet capture in the next days. Let's do it correctly.

To sum up:

debug log
packet capture

Anything else ?

burmistrzak commented 4 months ago

@chris-1243 Ha, no worries! Here's my shopping list for you:

Packet capture
Debug logs
Devices involved (model, firmware)
Procedure (when, what, how)
Timestamps (if possible)

Also would be interesting to see, if anything changes when you switch the PTM216Z into a command-based mode (e.g. GreenPower_2). That's all. 😉

I saw a few interesting things already, but we'll have to wait for more data to draw any conclusions.

[2024-06-14 18:51:11] debug:    zh:zstack:unpi:parser: --- parseNext []
[2024-06-14 18:51:12] debug:    zh:zstack:unpi:parser: <-- [254,3,68,128,205,242,8,240]
[2024-06-14 18:51:12] debug:    zh:zstack:unpi:parser: --- parseNext [254,3,68,128,205,242,8,240]
[2024-06-14 18:51:12] debug:    zh:zstack:unpi:parser: --> parsed 3 - 2 - 4 - 128 - [205,242,8] - 240
[2024-06-14 18:51:12] debug:    zh:zstack:znp: AREQ: <-- AF - dataConfirm - {"status":205,"endpoint":242,"transid":8}
[2024-06-14 18:51:12] debug:    zh:zstack:unpi:parser: --- parseNext []
[2024-06-14 18:51:12] debug:    zh:zstack: Data confirm error (null:1282,205,4)
[2024-06-14 18:51:12] error:    zh:controller:greenpower: Error: Data request failed with error: 'No network route' (205)
[2024-06-14 18:51:12] debug:    zh:controller: Data is from unknown device with address '33501', skipping...

Edit: Might be appropriate to open a new issue under https://github.com/Koenkk/zigbee-herdsman for the PTM216Z/Hue thing.

chris-1243 commented 4 months ago

@burmistrzak

Packet capture Debug logs

Pairing_PTM216Z_failed_2024.06.15.zip

Devices involved (model, firmware)

Philips 9290022166, firmware 1.116.3

Procedure (when, what, how)

I tried to pair the PTM216Z via NFC and via the standard procedure (old the channel button for more than 7s and wait in Z2M if the device is recognized). Both attempt failed with a no network route error (205).

Also would be interesting to see, if anything changes when you switch the PTM216Z into a command-based mode (e.g. GreenPower_2).

Unfortunately and via the IOS app, I am not able to make those changes. I may only choose the channel, commissioning mode and send the pairing frame.

Edit: I will let you check what I just sent and then open an issue

burmistrzak commented 4 months ago

@chris-1243 Thanks! Much better this way. 😊

I have an idea, but need a successful pairing to compare. Can you provide another set of packets + logs, but use a PTM215Z this time around? Everything else should stay the same (Hue light used, etc.), if possible.

It should be possible to configure the PTM216Z device mode using a generic NFC tool, but we don't know what commands to send... Can you try contacting EnOcean for help with this?

chris-1243 commented 4 months ago

Can you provide another set of packets + logs, but use a PTM215Z this time around? Everything else should stay the same (Hue light used, etc.), if possible.

Pairing_PTM215Z_2024.06.17.zip

There it is... Everything remained the same (same RPI4, coordinator, Hue router, Z2M version...) except the PTM used as requested.

It should be possible to configure the PTM216Z device mode using a generic NFC tool, but we don't know what commands to send... Can you try contacting EnOcean for help with this?

The email is on its way. Let's see the answer from EnOcean. I downloaded a generic NFC app for iOS. I need to be sure which commands I should send. Otherwise, my PTM216Z might be broken...

I created two separate PR in the documentation. The first to get the latest user manual and a second to add some tips regarding NFC pairing capability.

burmistrzak commented 4 months ago

@chris-1243 Ok, so Z2M is not responding with a GP Pairing, because we get a 'No network route' (205) error beforehand. This might be caused by:

another hue firmware bug
the PTM216Z device mode configured
something completely different

Can you please provide another round of packets & logs? This time however, please use the PTM216Z again, but pair it with a Ikea LED2101G4, instead of a Hue fixture. Also, which of the Hue lamps you've tested have a Green Power input and output cluster?

chris-1243 commented 4 months ago

As I was trying to add some routers in your table I found IKEA LED2103G5 is able to translate ZGP frame. So, as I do not use it for the moment, I picked this one to do some tries. I do not have to remove a device from my main network.

This IKEA device as both input/output cluster on endpoint 242. Pairing a PTM216Z just failed but the PTM215Z paired correctly in unicast mode.

Also, which of the Hue lamps you've tested have a Green Power input and output cluster?

Pairing successful:
- 4034031P7, input/output cluster
- 8718696548738, input/output cluster
Pairing failed:
- 5047130P9, only output cluster
- 9290022166, only output cluster

Pairing_PTM21xZ_ikea_led2103G5_2024.06.18.zip

You will find in this*.zip file a debug log and two captures. One regarding the PTM215Z pairing and the second regarding the failed attempt of the PTM216Z. Both tries were done via IKEA LED2103G5.

burmistrzak commented 4 months ago

@chris-1243 Oh, that's so interesting! I first thought it might have something to do with the additional GP Sink cluster, but after seeing your results from the IKEA bulb, I'm not so sure about that.

Another option could be that manufacturers simply do not test/certify Green Power switches in generic mode with their products... 🤔

@Koenkk I assume such an error 👇 is originating from a Zigbee router, and not the coordinator itself?

[2024-06-14 18:51:12] debug:    zh:zstack:znp: AREQ: <-- AF - dataConfirm - {"status":205,"endpoint":242,"transid":8}
[2024-06-14 18:51:12] debug:    zh:zstack:unpi:parser: --- parseNext []
[2024-06-14 18:51:12] debug:    zh:zstack: Data confirm error (null:1282,205,4)

burmistrzak commented 4 months ago

@chris-1243 Also, and please correct me if I'm wrong here, but it seems to be impossible to pair the PTM216Z as unicast? If you get it paired, then only in broadcast/groupcast mode, right?

burmistrzak commented 4 months ago

@chris-1243 Another difference between the PTM21x models is the value of the Fixed Location options field in the initial Commissioning frame. The PTM216Z has this field set to false, for some reason. Let me quote the GP specification:

The FixedLocation sub-field is a Boolean flag. If the value of this sub-field is 0b0, then it indicates that the GPD can change its position during its operation in the network. If the value of this sub-field is 0b1, then the GPD is not expected to change its position during its operation in the network.

Not sure if this is only informational or actually has some influence on the pairing process tho... 🤔

chris-1243 commented 4 months ago

Also, and please correct me if I'm wrong here, but it seems to be impossible to pair the PTM216Z as unicast? If you get it paired, then only in broadcast/groupcast mode, right?

This is completely correct. Each time I have been able to pair the PTM216Z, it was only in broadcast/groupcast mode. I tried several different routers and except the two mentioned above from Philips, none have been able to achieve a successful pairing.

I would be really curious to see how a PTM215ZE is pairing to a network. It should be a greenpower_2 and I would assume is acting like the PTM215Z. This an assumption only...

chris-1243 commented 4 months ago

I just had a look in the PTM215ZE and the default ZGP Device ID used is 0x02 (ZGP ON/OFF Switch). While pairing the good old PTM215Z, it is sending the exact same Device ID.

Untitled

On the other hand, the PTM216Z is using ZGP Device ID 0x07 (Generic). I guess, as you already pointed, there might be something to dig in.

Untitled_02

chris-1243 commented 4 months ago

PTM 215ZE uses by default the command-based device model “ON / OFF Switch” identified by Device ID 0x02. PTM 215ZE can be configured via NFC to use one of the other command- based device models or the Generic Switch device model via the NFC PC configuration tool as described in chapter 4.2.

I need to find an NFC reader/writer in order to be able to change the Device ID....

it is not written the same in the PTM216Z user manual.

By default, PTM 216Z uses the Generic Switch model identified by Zigbee Green Power Device ID 0x07. It is possible to select a different (command-based) model via NFC as described in chapter 4.2.

burmistrzak commented 4 months ago

@chris-1243 Thanks for confirming! 🙌

The DeviceID i.e. device mode really seems to play some kind of role here.

It's great that EnOcean mentions the NFC chapter in the manual, but fails to provide the actual commands required. Lol...

Maybe give this app a try: https://apps.apple.com/us/app/nfc-tools/id1252962749

burmistrzak commented 4 months ago

Packet No. 1747 is unicast, so IKEA is actually following the specification. This means Philips Hue likely messed up something in firmware. Not good...

I'll open a ticket with Signify, but if anyone has a contact there, let me know. 🤞

Just heard back from them... Unfortunately, I received a more or less boilerplate response for feature requests (?), and the ticket got closed. Lovely. 🙄

We'll probably have to find another way to get that message to the right people.

chris-1243 commented 4 months ago

Maybe give this app a try: https://apps.apple.com/us/app/nfc-tools/id1252962749

I have it as well but the main problem is what kind of NFC commands should I write not to brick my PTM216Z and to change the ZGP device ID

burmistrzak commented 4 months ago

I have it as well but the main problem is what kind of NFC commands should I write not to brick my PTM216Z and to change the ZGP device ID

That's the price question! 😅 Hopefully EnOcean is replying to your inquiry. 🤞

chris-1243 commented 4 months ago

I just had a look there https://github.com/Koenkk/zigbee-herdsman-converters/blob/master/src/converters/fromZigbee.ts and in https://github.com/Koenkk/zigbee-herdsman-converters/blob/master/src/devices/enocean.ts. If we are able to change the ZGP device ID, the PTM216Z might be seen as a PTM215ZE in Z2M as they share the same ieeeAddr.

Z2M uses Device ID and ieeeAddr to do the difference.

Hopefully EnOcean is replying to your inquiry. 🤞

Wait and see....

burmistrzak commented 4 months ago

I just had a look there https://github.com/Koenkk/zigbee-herdsman-converters/blob/master/src/converters/fromZigbee.ts and in https://github.com/Koenkk/zigbee-herdsman-converters/blob/master/src/devices/enocean.ts. If we are able to change the ZGP device ID, the PTM216Z might be seen as a PTM215ZE in Z2M as they share the same ieeeAddr.

Uh, you're right! Maybe a dedicated definition for the PTM216Z in GreenPower_2 mode is necessary? Not sure if the supported actions are exactly the same as with an actual PTM215ZE... 👀

chris-1243 commented 4 months ago

Not sure if the supported actions are exactly the same as with an actual PTM215ZE... 👀

According to the two user manuals, if configured as a ZGP device ID 0x02, commands seem to be the same.

samuele2723 commented 4 months ago

thank you! this post saved my time to understand how were unresponsive those switch

burmistrzak commented 4 months ago

thank you! this post saved my time to understand how were unresponsive those switch

No, thank you! Without you opening the issue, I probably wouldn't have looked into this whole situation. 😉

burmistrzak commented 4 months ago

Alright, I've contacted Philips Hue developer support, and thoroughly explained the situation. Maybe we'll reach the right folks that way... 🤞

chris-1243 commented 4 months ago

So, I am still waiting an answer from EnOcean.

By the time, I have found an usb NFC reader/writer. I hope I could use it with the PC based software and then be able to change the ZGP device ID configuration.

burmistrzak commented 4 months ago

@chris-1243 You'll have to register to download the NFC software... 🙄 https://www.enocean.com/en/product/enocean-nfc-configurator

AFAICT, it's designed to work with a specific model of USB NFC reader, but maybe a generic one will work as well? Otherwise, we might have to take to software apart, and find the required commands that way... 😏

Koenkk / zigbee2mqtt