search

openhab / org.openhab.binding.zwave

openHAB binding for Z-Wave
Eclipse Public License 2.0
171 stars 202 forks source link

ZEN16 not reporting manual SW events due to missing COMMAND_CLASS_BASIC property #1632

Closed dw-8 closed 3 years ago

dw-8 commented 3 years ago

Since OH3 was updated in early 2021, zen16_1_0.xml had removed COMMAND_CLASS_BASIC, so OH is no longer reporting events whenever the manual switches (SW1, SW2, and SW3) are used. Instead, the state of each switch/relay is only getting updated via polling.

Comparing OH3 with OH2.5, 

<property name="binding:*:OnOffType">COMMAND_CLASS_SWITCH_BINARY:1</property>
 <property name="binding:*:OnOffType">COMMAND_CLASS_SWITCH_BINARY:2</property> 
 <property name="binding:*:OnOffType">COMMAND_CLASS_SWITCH_BINARY:3</property>

should be:

 <property name="binding:*:OnOffType">COMMAND_CLASS_SWITCH_BINARY:1,COMMAND_CLASS_BASIC:1</property>
 <property name="binding:*:OnOffType">COMMAND_CLASS_SWITCH_BINARY:2,COMMAND_CLASS_BASIC:2</property>
 <property name="binding:*:OnOffType">COMMAND_CLASS_SWITCH_BINARY:3,COMMAND_CLASS_BASIC:3</property>

I made the above changes and it fixed my event reporting issue that was introduced when I upgraded from OH2.5 to OH3.0 and OH3.1.

Also, the following eight extraneous channels were incorrectly added at the same time, and should be removed as they do not exist on the ZEN16.

      <channel id="switch_binary" typeId="switch_binary">
      <channel id="switch_dimmer" typeId="switch_dimmer">
      <channel id="scene_number" typeId="scene_number">
      <channel id="sensor_binary" typeId="sensor_binary">
      <channel id="meter_kwh" typeId="meter_kwh">
      <channel id="meter_kvah" typeId="meter_kvah">
      <channel id="meter_watts" typeId="meter_watts">
      <channel id="battery-level" typeId="system.battery-level">

The database record should also be updated to remove the extra channels: https://opensmarthouse.org/zwavedatabase/1326

Also, if possible, zen16_1_0.xml should really be split into separate configuration files for version 1.20 (aka v1.02 per Zooz) and version 1.30-1.40 (aka v1.03 and v1.04 from Zooz), so users with version 1.20 do not have extra invalid parameters. Version 1.20 introduced parameters 15-20, while version 1.30 introduced parameters 21-24. version 1.40 did not introduce any new parameters.

cdjackson commented 3 years ago

@bwosborne2 Are you able to verify this? I'm not sure if you have one of these?

bwosborne2 commented 3 years ago

I do not have one but @5iver has one. He did the initial entry, I believe. If you want me to get one, I can try.

cdjackson commented 3 years ago

From what I can tell on the web (at least Amazon) they aren't available now and I don't see if on the zooz site. I see some of the changes like removing the battery CC initially makes sense, but sometimes these devices have secondary functions. I'm sure @5iver didn't make such a mistake, but maybe someone else modified the database since.

I'll have a sift back through history to see what changes were made...

dw-8 commented 3 years ago

The Zooz product page is weird; they don't even list their new 700 Series ZEN17 universal relay either.

The ZEN16 Multi-Relay are still sold and listed on thesmartesthouse.com.

FYI... I contacted Zooz support on Jun 30, regarding the event notification, and their response said they were considering removing OpenHab from the compatibility list as "their seems to be many issues with the new {OH3] platform and haven't been able to receive any feedback from their [OpenHab] team.

Based on their response, I went back and compared the xml file from a copy of the old 2.5 codebase I had downloaded last fall with the current version, and found the key difference noted above regarding COMMAND_CLASS_BASIC.

Note: my ZEN16 is running firmware 1.02 (which is 1.20 in OH).

Zooz sent me a copy of firmware 1.03 and 1.04 just in case, but I didn't need to update as adding COMMAND_CLASS_BASIC fixed my event notification issue using 1.02.

Also, per Zooz, the following bug fixes were made in firmware 1.04.

  1. When the hub sends binary_switch_get command to the endpoint that just reported ON using the binary switch report, the the ZEN16 replies with the opposite state causing the status to report incorrectly on the hub if the hub polls the ZEN16.
  2. If parameter 1 is set to value 1 (default) and relays are turned on from individual endpoints (rather than master) and pwer is cut off when the relays are on, then the relays turn back to OFF instead of on once power is restored.
dw-8 commented 3 years ago

you can also compare the differences between zen16_0_0.xml (for version 1.0) with zen16_1_0.xml (for versions >1.0).

bwosborne2 commented 3 years ago

From what I can tell on the web (at least Amazon) they aren't available now and I don't see if on the zooz site.

Here is is on their store site. Discounted though. https://www.thesmartesthouse.com/products/zooz-z-wave-plus-s2-multirelay-zen16-with-3-dry-contact-relays-20-a-15-a-15-a

cdjackson commented 3 years ago

The ZEN16 Multi-Relay are still sold and listed on thesmartesthouse.com.

Ok, thanks. I didn't look there - I looked on getzooz.com (I'm not sure who runs what to be honest).

their seems to be many issues with the new {OH3] platform and haven't been able to receive any feedback from their [OpenHab] team.

I'm not really sure what that means to be honest or who "their team" are?

Anyway, as mentioned, I guess someone needs to make the changes in the database - merging the PR here will not work as it will be overwritten again within a week at the most.

dw-8 commented 3 years ago

The ZEN16 Multi-Relay are still sold and listed on thesmartesthouse.com.

Ok, thanks. I didn't look there - I looked on getzooz.com (I'm not sure who runs what to be honest).

their seems to be many issues with the new {OH3] platform and haven't been able to receive any feedback from their [OpenHab] team.

I'm not really sure what that means to be honest or who "their team" are?

Anyway, as mentioned, I guess someone needs to make the changes in the database - merging the PR here will not work as it will be overwritten again within a week at the most.

Zooz was referring to the OpenHab developers not responding to them, though I didn't see any recent issues logged in github for ZEN16, but there was a thread in the Community where no developer responded back or resolved the issues, and this is partly related to my issue, which was fixed by adding COMMAND_CLASS_BASIC property. The other part of that thread I believed was fixed with firmware 1.04 https://community.openhab.org/t/zen16-strange-behavior/116631.

I just created an ID and created a ticket to gain access to update the zwave database.

Looking back at the commits... It looks like firmware 1.02 was added to OH2.5 in commit #1373 using the original zen16_0_0.xml, which is when it was last working correctly.

On Dec 17, 2020, commit #1468 OH2.5 / #1467 OH3.0 was when it was first broken. In this commit, v1.02 was removed from zen16_0_0, and zen16_1_10 was created for v1.02, however, it was not created correctly in the database, as it used manufacturerRef A000:A055, instead of A000:A00A, and added 8 extra channels, and removed the COMMAND_CLASS_BASIC property.

On Dec 28, 2020, commit #1481 OH2.5 / #1482 OH3.0 added manufacturerRef A000:A00A back to zen16_1_10, which fixed the issue reported in the community https://community.openhab.org/t/zooz-zen16-multirelay-v1-2-not-recognized/111658

On Jan 2/4, 2021, commit #1491 OH2.5 / #1488 and #1495 OH3.0 removed manufacturerRef A000:A055, and renamed zen16_1_10 to zen16_1_0

On Feb 9/10, 2021, commit #1529 OH2.5 / #1532 OH3.0 removed versionMax to allow firmware v1.04.

cdjackson commented 3 years ago

Zooz was referring to the OpenHab developers not responding to them

Again - I don't really know what that means? Who exactly are they talking to? I'm the main developer of the ZWave binding, so if by "OpenHAB developers" they mean for the Wave binding, then I've not seen anything. I've not received anything to my personal email, they've not opened any tickets either on GitHub or the opensmarthouse.org ticket system, and I've not seen anything obvious from them on the forum.

If I've missed something, then I can only apologise. However I would suggest that they really should be more specific about who they are talking to. Another option is they are using one of the other maintainers emails - I'm just not sure - sorry.

there was a thread in the Community where no developer responded back or resolved the issues

Sorry - I didn't realise that this was from Zooz, but at that time I was in the middle of moving from the UK to New Zealand and was not especially active. I thought from your comment above that the developers weren't responding to them that they had sent more than one comment on the forum.

bwosborne2 commented 3 years ago

Who exactly are they talking to

I suspect Scott and I? I know I let them know I was pulling back from OH. Perhaps Scott did the same. I did not present myself as a developer though. I can start an email thread to connect you if you wish.

Agnes especially is top notch.

cdjackson commented 3 years ago

I can start an email thread to connect you if you wish.

I think probably the best thing is if they open a ticket if they have issues. Since I'm not in the US I'm unable to use their devices so it may well require someone else to provide input (as we see here with @dw-8 kindly helping out). At least if they open a ticket publicly, then others can help.

bwosborne2 commented 3 years ago

Since I'm not in the US I'm unable to use their devices If you wish I can try to get one & test it. It at least gives another viewpoint but the decision is yours.

brianlay714 commented 3 years ago

I recently bought this device but haven’t installed it yet. I’ll install it next week and can them help with testing

Sent from my iPhone

On 12 Jul 2021, at 15:21, bwosborne2 @.***> wrote:

 Since I'm not in the US I'm unable to use their devices If you wish I can try to get one & test it. It at least gives another viewpoint but the decision is yours.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

bwosborne2 commented 3 years ago

I may be able to assist with the database piece if @dw-8 is not comfortable with that. I have done a few over the past couple of years.

dw-8 commented 3 years ago

I am just waiting for Chris to grant me write access to update the database. I will let you know if I have any problems.

bwosborne2 commented 3 years ago

OK good idea.

cdjackson commented 3 years ago

I am just waiting for Chris to grant me write access

Hmm - strange. You raised a ticket and I replied and updated this yesterday, and you responded to this ticket - so I assume you received the notification that this was done already?

dw-8 commented 3 years ago

I did get and replied back to you on that ticket but my access wasn't updated. I still don't see an edit option.

cdjackson commented 3 years ago

You definitely have edit access. Maybe you need to log out and log back in for this to update - I'm not sure. I would have expected that your system would log you out anyway given it's been nearly 24 hours or so.

bwosborne2 commented 3 years ago

The Edit button id very low contrast.

image

dw-8 commented 3 years ago

I have edit access now. I have started editing the database.

To do this correctly, I was planning on trying to split the database into 3-4 different versions (<1.1, 1.1, 1.2, >1.2), as new parameters were added with each firmware, as per Zooz change log https://www.support.getzooz.com/kb/article/356-zen16-multirelay-change-log/.

However, when I look at Inovelli's devices in the database, they are all defined with ALL versions despite Inovelli issuing lots of firmware updates; and folks are just not creating new separate versions and just adding new features/changes to the existing version, so it just means end users will have parameters that do not match up. Should I bother splitting them up?

I am not entirely sure of the Min and Max versions with respect to how it decides <, =, and >.

Right now, there are two versions of ZEN16 in the database; one is versions <1.0 and other is versions >1.0. Since the initial firmware was 1.00, doesn't that mean the version that is versions <1.0 is never actually used and should really be changed to <1.1? I assume the first one should be changed from Version (Max) from 1.0 to 1.1. The second record would be changed to have Version (Min) and Version (Max) both set to 1.1 to cover firmware 1.01 (and remove the extra parameters introduced in v1.02+). The third database record (to be created) would have Version (Min) and Version (Max) set to 1.2 to cover firmware 1.02. The fourth database record (to be created) would have Version (Min) set to 1.3 with no Version (Max) to cover firmware 1.03 and 1.04. Is this correct?

To make the two new database records (for v1.02 and v1.03+), I planned on modifying my Zwave jar in Eclipse to remove the existing ZEN16 thing so that OpenHab generates a clean Unknown base XML file that I can use to upload into the database and then modify that as needed afterwards.

cdjackson commented 3 years ago

You can find the comparison here -:

https://github.com/openhab/org.openhab.binding.zwave/blob/main/src/main/java/org/openhab/binding/zwave/internal/ZWaveProduct.java#L86

It's up to you if you want to create multiple versions. Of course, this is best and avoids errors which will otherwise be thrown if there are missing parameters.

To make the two new database records (for v1.02 and v1.03+), I planned on modifying my Zwave jar in Eclipse to remove the existing ZEN16 thing so that OpenHab generates a clean Unknown base XML file that I can use to upload into the database and then modify that as needed afterwards.

I'm not sure what you are doing here. You should not mess with the binding - there is no need. The XML that OH creates is completely different to the one inside the binding and they are completely unrelated.

dw-8 commented 3 years ago

I was going to remove the two zen16 xml things so that the Zwave binding doesn't know about ZEN16, then delete and re-add the thing in OH.

cdjackson commented 3 years ago

I was going to remove the two zen16 xml things so that the Zwave binding doesn't know about ZEN16, then delete and re-add the thing in OH.

This will make no difference to anything. As I said above, the XML files that are inside the binding are not related or linked in any way to the ones that OH creates, and that are then used to create the database entries.

dw-8 commented 3 years ago

I left DBid 1176 unchanged for firmware v1.00, and modified DBid 1326, and created two new devices (DBid 1399 and 1400).

For DBid 1326, there are 4 parameters (21-24) that need to be deleted and a bunch of classes within Endpoint 0 need to be deleted. I made the endpoint changes to match DBid 1176. I changed this device to be limited to just v1.01 now. According to Zooz change log, v1.01 was only released as an OTA update, so not many OH users probably did an OTA update to have this specific firmware version, so all these changes and fixes to this version may not be used by anyone, as they would have already did another OTA update to v1.02 or higher.

For DBid 1399, there are 4 parameters (21-24) that need to be deleted as they were copied over from DBid 1326. This version for v1.02 was created from the xml file for my existing device that has firmware v1.02, so the Endpoints created should be correct.

For DBid 1400, I uploaded too many attachments and cannot delete the incorrect attachments. I uploaded the manual twice and named one of them as the image, and uploaded the image twice. There should only be 3 attachments: the v1.03 advanced settings, the v1.04 manual, and the image. I used my v1.02 xml file and copied the info from DBid 1399 to create this device for v1.03 and higher.

I am not sure what the image label should be since it doesn't show a preview of the image for any of the revision devices and only shows the image when looking at DBid 1176. I am referring to when you are searching the database, and it shows a brief summary of the device info in the right side panel with an image of the device.

5iver commented 3 years ago

Zooz was referring to the OpenHab developers not responding to them

Again - I don't really know what that means? Who exactly are they talking to?

Hello, Chris! Good to see you... hope your roots are growing back in :evergreen_tree:! The Smartest House contacts me about every month when they release a new device or have customers using OH with issues. I've suggested several times that someone from the customer service team should join the forum to front issues customers are having, especially if the customers themselves won't post on their own. I've also asked to be notified when new firmware is released, so that I can create new database entries, but this never happens. I was asked to look into the ZEN16 on OH3 due to reports of issues by their customers, and I let them know that I am very busy with other things, but that I would look into it eventually. I had setup an OH3 test environment to look into it and come up with the same conclusions as @dw-8 in his excellent assessment (:+1:). My original database entry had been messed up and needed to be split into several entries for each firmware version. As often happens, I make a device entry, someone purchases the device with updated firmware, and they update the entry instead of creating a new one for the updated firmware. The database updates often are approved before I have a chance to review them. I had still planned to make the needed corrections to this device's database entry, but I can also assist if anyone else would like to get to it before I can!

Sorry for being so slow in responding and getting this cleaned up. OH is very low in my priorities.

Perhaps Scott did the same. I did not present myself as a developer though.

Yes, I informed Agnes of the events that occurred prior to the 3.0 release and that I had been focusing on other things. 13 chickens (they just started to make more this month!), 5 turkeys, we'll add two pigs in August, two bee hives, we're always adding more things to the orchard, and there are vegetables popping up all over the place! I think I can call myself a farmer now!

bwosborne2 commented 3 years ago

Hey Scott, sorry to tag you on this. I have to wonder that perhaps Zooz support is not notified when new firmware is released until they get a support ticket. I have been busy in real life and slowly moving away from OH. I have been testing another system but my main one is still on the ancient 2.5.2 version of OH.

cdjackson commented 3 years ago

Hey Scott - great to hear from you 👍

hope your roots are growing back

Yeah - slowly! It's been a lot more time consuming that we thought - selling everything in the UK, and then having to buy everything in NZ - the stress of selling and buying houses (not in that order) and having multiple mortgages etc has been "fun", but we're getting there now (at last).

Sorry for being so slow in responding and getting this cleaned up. OH is very low in my priorities.

Hey - no problem at all - I completely understand (and have been in the same situation myself over the past 6 months due to the big move). I also agree that it makes sense for Zooz to either get on the forum, or open an issue here - if they aren't willing to do that then I don't really think they should complain when their devices aren't working properly...

13 chickens

Ah - we only have 12! :)

...

Anyway... I will take a look at the changes that @dw-8 has made later today. Many thanks for following this up @dw-8.

bwosborne2 commented 3 years ago

13 chickens

Ah - we only have 12! :)

I just joined that club this year with 6. In the country, babysitting the neighbour's kid takes on a whole different meaning. We had a baby goat living in our fenced yard for a while.

cdjackson commented 3 years ago

I just joined that club this year with 6.

Yes - same here. In the UK some of our neighbours has chickens, but we didn't. Ours came with the house - although it's mid winter here so we've not really had too much return from them yet...

One of our neighbours is a "real" farm - they have sheep and a lot of lambs at the moment. The other neighbours are lifestyle blocks and have alpaca and goats. It's fun :)

5iver commented 3 years ago

This is all new to us too! We started with 5 layers in August, added 5 broilers in March (they threw in 3 extras), then picked up 5 turkeys and 2 nuc bee hives in May. We have a very broody hen right now with 12 eggs under her, so we should have a bunch more in a couple weeks! We are in the process of clearing a 100x100 area in our woods of thorny, poisonous, and invasive plants, as well as LOTS of standing dead trees (old elms and hemlocks), to fence in for a 'pasture in the woods'. The pigs are going on the other side of the property in another area in the woods. We've thought about goats, but they seem too fiesty and destructive, so we had decided to try some sheep in the spring!

I had no idea you guys were into this kind of stuff! I guess you really can understand why I haven't been around as much! Sorry for being off-topic, @dw-8!

dw-8 commented 3 years ago

I just tried the Export OH2 XML option for the 2 new database records (1399 and 1400) that I created from scratch, and the channels are still missing the COMMAND_CLASS_BASIC:1/2/3 property, for endpoints 1-3, so my edits are not going to fix this issue. DBid 1326 is still missing the property as well, but I assumed that was because I can't delete items. I just made some edits for DBid 1176, and it still correctly has the COMMAND_CLASS_BASIC:1/2/3 properties.

What changes do I need to make to the records, so that COMMAND_CLASS_BASIC:1/2/3 are included in the endpoints?

dw-8 commented 3 years ago

I just figured it out.

You need to edit COMMAND_CLASS_SWITCH_BINARY_V1 command class for each endpoint, and set Basic to True.

The import from the OH XML does not set this option.

cdjackson commented 3 years ago

The import from the OH XML does not set this option.

No - this must of course be set manually. There's no way to know this automatically.

dw-8 commented 3 years ago

Is this a common setting needed for physical switches to send events when manually activated?

If it is, then maybe the guides need to be updated to mention this, as this would be easy to overlook, and cause "bugs" in new/updated devices that are added to the database, which is what probably happened with this device.

The person changed the manufacturerRef to a random number (as per the guide), imported the OH XML, copied the overview, parameters, and associations, from the original DB, then added the additional parameters, and uploaded the reference materials (manual and image), but forgot to change the manufacturerRef back to the correct value, and did not know to edit the COMMAND_CLASS_SWITCH_BINARY_V1 command class for each endpoint, and set Basic to True.

Obviously, I just did the same steps as above, but remembered to change the manufacturerRef back, and it would have been pushed out to users with this property not set to True.

cdjackson commented 3 years ago

Is this a common setting needed for physical switches to send events when manually activated?

It's nothing to do with manual activation - it's just down to how the device works. It is heavily dependant on the device - the user needs to know what the device does - it's not possible to do this automatically. It's not easy to document everything here.

dw-8 commented 3 years ago

Ok, maybe nothing to do with manual activation, but needed for most switches in general.

I just took a quick look at Aeotec, Inovelli, Leviton, and Zooz, and most (if not all) of their switches had COMMAND_CLASS_BASIC property set. And, the few that don't have it may have this same event notification "bug" that haven't been reported by users.

cdjackson commented 3 years ago

Ok, maybe nothing to do with manual activation, but needed for most switches in general.

Yes, for switches it is normally set - but not always. We could set it by default, and then remove it when it causes problems. Or we can not set it, and add it - there's no way to know. For other devices, it is often not set.

And, the few that don't have it may have this same event notification "bug" that haven't been reported by users.

If you have these devices, and there are issues, then please feel free to update, but please do not simply go around setting this as it can also cause problems!

bwosborne2 commented 3 years ago

Yes, for switches it is normally set - but not always. We could set it by default, and then remove it when it causes problems. Or we can not set it, and add it - there's no way to know. For other devices, it is often not set.

Does the Z-Wave Plus specification require this or is it optional?

dw-8 commented 3 years ago

I am sure you are busy, but any idea when you will get around to reviewing the changes (and deleting the marked for deletion items) made to the 4 ZEN16 database records?

As a reminder, the critical fix needed was adding back the missing COMMAND_CLASS_BASIC property to the endpoints.

The rest of the changes were splitting into different versions, removing the extraneous channels that don't exist on the ZEN16, and tweaking the parameters so that they are more descriptive (and limit to the options filter).

dw-8 commented 3 years ago

@cdjackson approved the changes as a part of database updates #1636:

I verified these changes that were included in the build 88 zwave binding jar file fixed the issue where the ZEN16 does not report manual switch events due to missing COMMAND_CLASS_BASIC property, so I am closing this issue https://ci.openhab.org/job/openHAB-ZWave/88/artifact/target/org.openhab.binding.zwave-3.2.0-SNAPSHOT.jar

Note

5iver commented 3 years ago

I have removed the Channels that were marked for deletion in 1326. I tried to remove the Command Classes too, but they did not go away (I did not see and error). This may be a bug in the website. I also removed the configuration parameters marked for deletion in 1326 and 1399. I also saw switch_binary Channels in Endpoint0 for 1400 and 1399, so I removed them.