Improvements of the Z-Wave binding for the 1.4.0 version of openhab

[GoogleCodeExporter]

With the (upcoming) release of 1.3.0, it's time to think about and work on the 
next version of the Z-Wave binding. We like this as much to be a collaborative 
effort as the previous version was.

Ideas and suggestions about what to implement are welcome. In case you actually 
want to start coding, there is a list of items to work on below:

high priority:

- D make binding between command classes and items more dynamic. So that 
switches can be bound to dimmers, sensors to contacts or switches, etc.
- M initialize / read binding values in the DYNAMIC node stage during binding 
initialization.
- M Make polling a binding action, so that it is configurable from the items 
file. Default should be no polling.
- D Implement associations node stage to read associations. These can be used 
to query associated nodes for their status, in case the associated nodes do not 
report themselves directly (don't know if this is neccessary).
- M Implement some form of serialization to save / cache the configuration of 
the binding to speed up initialization.
- E Implement the SWITCH_TOGGLE_BINARY command class.
- E Implement the SWITCH_TOGGLE_MULTILEVEL command class.
- E Implement THERMOSTAT_MODE command class. 
- E Implement THERMOSTAT_SETPOINT command class.
- E Implement BASIC_WINDOW_COVERING command class.
- E Implement DOOR_LOCK command class.
- E Implement ALARM command class.
- E Implement LOCK command class.
- M Implement MULTI_CMD command class. Used to group multiple commands 
together. Useful for Battery operated devices to empty the wake-up queue into 
one
command at once.
- E Implement SIMPLE_AV_CONTROL command class.
- DD Implement SECURITY Little information. Would require changes to 
encapsulation, secure key storage in openhab. Not much information. Very useful 
for locks though. One of the weaker points of Z-Wave that security was an 
afterthought.

normal priority:

- M Make binding actions for alarm types, so that binding can be done on a 
specific alarm. Remove alarm values from the binding again then, to make it 
stateless.
- D Handle adding nodes to the network / removing nodes from the network.
- D Control node configuration parameters using a binding. (Requires changing 
binding item string format to add parameter number or something).
- DD Implement neighbors node stage. Request node neighbors and update return 
routes based on the neighbour list and rtt values.
- M Implement the APPLICATION_STATUS command class. Handle incoming rejected 
and busy signals.
- D Implement the SWITCH_ALL command class. handle incoming switch all commands 
and implement sending. Some discussion on what to do on those incoming signals 
and when to send would be neccesary.
- D Implement the SCENE_ACTIVATION command class. Some discussion on how to map 
scenes onto the openhab bus and vice versa would be neccessary.
- E Implement METER_PULSE command class.
- M Implement CLIMATE_CONTROL_SCHEDULE class. Requires some discussing about 
how to map / handle scheduling capabilities to open-zwave.
- D Implement SCHEDULE_ENTRY_LOCK command class. Limited info on the command 
class. Would probably require the a device, commercial software to control it, 
and serial monitoring software.
- E Implement POWERLEVEL command class. Setting this too low, will make 
communication unreliable!
- E Implement PROTECTION command class.
- E Implement NODE_NAMING command class. Would require discussion on how to 
implement: e.q. use item name, or just report or free settable.
- E Implement CLOCK command class.
- E Implement HAIL command class. Some devices use a HAIL to the controller to 
ask for polling.
- E Implement INDICATOR command class.
- E Implement PROPRIETARY command class. Would require discussion on how to 
define properietary payloads... in item string? as files?
- E Implement LANGUAGE command class.
- E Implement TIME command class.
- E Implement GEOGRAPHIC_LOCATION command class.
- D Implement COMPOSITE command class. Variation on Multi_Instance. Requires 
some changes to initialization and encapsulation etc.
- E Implement ENERGY_PRODUCTION command class.
- E Implement MANUFACTURER_PROPRIETARY command class. Would require discussion 
on how to define properietary payloads... in item string? as files?
- E Implement SCREEN_MD command class.
- E Implement SCREEN_ATTRIBUTES command class.
- E Implement SILENCE_ALARM command class.

low priority:

- E Make a reporting binding action for the node stage, create separate 
reporting binding actions for sleeping / dead.
- D Handle endpoints in multi-channel nodes that can be added /removed 
dynamically. 
- DD Handle the situation where the stick is not the primary controller. This 
can be in networks with another (commercial) controller system. In this case 
enabling the controller to be a SUC is advisable.
- DD implement bridging and virtual nodes. This allows items on the bus to 
become virtual nodes and use Z-Wave associations or Z-Wave scenes to use items 
on another bus directly in Z-Wave.
- D Separate the Z-Wave part from the binding part and make it a separate open 
source library like open-zwave.
- D Implement the CHIMNEY_FAN command class. Have not been able to find a 
device or documentation / code about it. Should be easy once found.
- D Implement MTP_WINDOW_COVERING command class.  Limited info on the command 
class. Would probably require the a device, commercial software to control it, 
and serial monitoring software.
- E Implement IP_CONFIGURATION command class. Only for reporting or also to set 
ip adresses?
- E Implement AV_CONTENT_DIRECTORY_MD command class.
- E Implement AV_RENDERER_STATUS command class.
- E Implement AV_CONTENT_SEARCH_MD command class.
- E Implement AV_TAGGING_MD(0x99,"AV_TAGGING_MD",null),
- D Implement TIME_PARAMETERS command class. Have not found any information.
- D Implement METER_TBL_MONITOR command class to read complex meters.
- D Implement THERMOSTAT_HEATING command class. Have not found any information.
- E Implement THERMOSTAT_OPERATING_STATE command class.
- E Implement THERMOSTAT_FAN_MODE command class.
- E Implement THERMOSTAT_FAN_STATE command class.
- D Implement THERMOSTAT_SETBACK command class. Have not found any information.
- D Implement DOOR_LOCK_LOGGING command class. Have not found any information.

ongoing:

- E Complete the list of device classes. This involves scanning software / 
documents for the required info or coming along an unknown device class on a 
device.

Original issue reported on code.google.com by jwsp...@gmail.com on 2 Sep 2013 at 12:45

[GoogleCodeExporter]

String ZwaveNode01HomeID    "Home ID [%s]" (gZwaveNode01) 
{zwave="1:1:command=info,item=home_id"}
Number ZwaveNode01NodeID    "Node ID [%s]" (gZwaveNode01) 
{zwave="1:1:command=info,item=node_id"}

Original comment by jwsp...@gmail.com on 1 Oct 2013 at 8:04

[GoogleCodeExporter]

I think so ben regarding dead, I will fix this.

Original comment by jwsp...@gmail.com on 1 Oct 2013 at 8:05

[GoogleCodeExporter]

I've fixed it. It's in my clone. I will provide binaries shortly. I've fixed 
some other bugs as well. 

I started working on saving and restoring of the configuration state as well. 
For now i've only implemented saving using XStream as XML serializer. Some 
example xmls are included below. Once I provide binaries I'm very interested in 
your XML's as well. They are saved in the etc/zwave directory.

Original comment by jwsp...@gmail.com on 1 Oct 2013 at 8:16

Attachments:

• node-xmls.zip

[GoogleCodeExporter]

I have been keeping up-to-date with your 1.4.0 fork - if I get the latest it 
should be stable? I am happy to build locally.

Original comment by ben.jone...@gmail.com on 1 Oct 2013 at 8:18

[GoogleCodeExporter]

oh man...I didn't use the underscore. Thanks. Works now.
For the siren I have to have a closer look.

Original comment by DerRo...@gmail.com on 1 Oct 2013 at 8:22

[GoogleCodeExporter]

Pretty stable... Although I want to run it overnight to be sure ;-)

Original comment by jwsp...@gmail.com on 1 Oct 2013 at 8:22

[GoogleCodeExporter]

I am willing to risk it ;). Will post my node files here once they are 
generated.

Original comment by ben.jone...@gmail.com on 1 Oct 2013 at 8:24

[GoogleCodeExporter]

I've made them interchangeable so that we can start sharing those in the same 
way as open-zwave does. I'll put in some checks on restoring so that invalid 
versions (both of the binding and node firmware) are not mixed.

Original comment by jwsp...@gmail.com on 1 Oct 2013 at 8:29

[GoogleCodeExporter]

And a binary for 1.3.1

Original comment by jwsp...@gmail.com on 1 Oct 2013 at 8:48

[GoogleCodeExporter]

:)
Could you please add it? :D

Original comment by DerRo...@gmail.com on 1 Oct 2013 at 8:51

[GoogleCodeExporter]

DerRocka, I don't see any switch_binary_set messages in your log for node 3. 
Are you sure you're actually toggling the switch? Can you provide a complete 
log (not just Z-Wave). 

Original comment by jwsp...@gmail.com on 1 Oct 2013 at 8:52

[GoogleCodeExporter]

Original comment by jwsp...@gmail.com on 1 Oct 2013 at 8:52

Attachments:

• org.openhab.binding.zwave-1.3.1-SNAPSHOT.jar

[GoogleCodeExporter]

Hmmm... Gotta find another way of distributing those. I'm out of space... quota 
exceeded.

Original comment by jwsp...@gmail.com on 1 Oct 2013 at 8:53

[GoogleCodeExporter]

Ok - node XML files being generated. They are all there apart from my door 
sensor (FGK101) as that only wakes up every hour. I will wait till that is 
generated and then post a full set. Looking good so far ;).

Original comment by ben.jone...@gmail.com on 1 Oct 2013 at 8:55

[GoogleCodeExporter]

Dropbox share?

Original comment by ben.jone...@gmail.com on 1 Oct 2013 at 8:57

[GoogleCodeExporter]

Might even taken two cycles so two hours :-)

Original comment by jwsp...@gmail.com on 1 Oct 2013 at 8:57

[GoogleCodeExporter]

Oh okay.

yes, of course. Well...I could quota exceeded. hmm...no idea right now.

I am toggling the switch item in HABdroid. But I have no rule or something like 
that. I thought that could be done with the item-definition itself. Like the 
other switches.

Original comment by DerRo...@gmail.com on 1 Oct 2013 at 8:59

[GoogleCodeExporter]

Yes, that should work but the event does not get to the binding somehow. It 
cannot send what it doesn't receive :-) 

Original comment by jwsp...@gmail.com on 1 Oct 2013 at 9:04

[GoogleCodeExporter]

True. But what could be the reason of that?

Original comment by DerRo...@gmail.com on 1 Oct 2013 at 9:06

[GoogleCodeExporter]

Cannot tell without a complete log. I do see node 2 toggled from habdroid, but 
that's probably another real switch.

Original comment by jwsp...@gmail.com on 1 Oct 2013 at 9:21

[GoogleCodeExporter]

yes it is another switch. can't find a switch_binary_set either. But can't 
upload at the moment. 

In my event.log I can see my item:
2013-10-01 23:27:16 - Alarm_Sirene received command ON
2013-10-01 23:27:25 - Alarm_Sirene received command OFF

At this time I should see something in openhab.log or in zwave.log shouldn't I? 
Nothing to see.

Original comment by DerRo...@gmail.com on 1 Oct 2013 at 9:33

[GoogleCodeExporter]

Could you post your items file?

Original comment by jwsp...@gmail.com on 1 Oct 2013 at 9:36

[GoogleCodeExporter]

Hmmm - think it might be a 'quota exceeded' for this issue, rather than us as 
users. I can't attach my node files either. I have emailed them to you directly 
JwS.

Original comment by ben.jone...@gmail.com on 1 Oct 2013 at 9:39

[GoogleCodeExporter]

Yes:

Switch Alarm_Sirene                     "Alarm-Sirene"          <siren-on>      
(gAlarmSystem)  { zwave="3:command=switch_binary" }
Number Alarm_Sirene_Battery             "Alarm-Sirene Batterie [%s]"            
(gZwaveStats)   { zwave="3:command=battery" }
Switch Tuer_Sensor                      "Tür Sensor [%s]"                      
 (gAlarmSystem)  { zwave="4:command=basic" }
//String Tuer_Sensor_Basic              "Tür Sensor [%s]"                      
 (gAlarmSystem)  { zwave="4:1:basic" }
Number Tuer_Sensor_Battery              "Tür Sensor Batterie [%s]"             
 (gZwaveStats)   { zwave="4:command=battery" }

/* Zwave Controller stick statistics */
Number ZwaveStatsSOF                            "Number Start of Frames[%s]"    
(gZwaveStats) { zwave="1:command=info,item=sof" }
Number ZwaveStatsACK                            "Number of ACK [%s]"            
(gZwaveStats) { zwave="1:command=info,item=ack" }
Number ZwaveStatsCAN                            "Number of CAN [%s]"            
(gZwaveStats) { zwave="1:command=info,item=can" }
Number ZwaveStatsNAK                            "Number of NAK [%s]"            
(gZwaveStats) { zwave="1:command=info,item=nak" }
Number ZwaveStatsOOF                            "Number of OOF [%s]"            
(gZwaveStats) { zwave="1:command=info,item=oof" }
.
.
.

Original comment by DerRo...@gmail.com on 1 Oct 2013 at 9:39

[GoogleCodeExporter]

Crap Google.

Original comment by jwsp...@gmail.com on 1 Oct 2013 at 9:40

[GoogleCodeExporter]

I'll create a new issue tomorrow with an updated todo. Now on a tablet.

Original comment by jwsp...@gmail.com on 1 Oct 2013 at 9:42

[GoogleCodeExporter]

Good idea. You get those node files ok?

Original comment by ben.jone...@gmail.com on 1 Oct 2013 at 9:43

[GoogleCodeExporter]

Yes. And it's getting late. So now I couldn't test it anyway :)

Original comment by DerRo...@gmail.com on 1 Oct 2013 at 9:44

[GoogleCodeExporter]

I've deleted attachments and it's still exceeded. jeez.

Original comment by jwsp...@gmail.com on 2 Oct 2013 at 8:10

[GoogleCodeExporter]

Like here:
https://code.google.com/p/support/issues/detail?id=30796
http://code.google.com/p/support/issues/detail?id=28358

A quota can be increased by "Project members"?

Original comment by DerRo...@gmail.com on 2 Oct 2013 at 8:19

[GoogleCodeExporter]

Man man...if you have a closer look at your line and my line of the zwave 
binding for the siren you can see, the only difference "<siren-on>". Without it 
everything works fine and I can hear the 100db too :)

Original comment by DerRo...@gmail.com on 2 Oct 2013 at 9:01

[GoogleCodeExporter]

Great to hear.

BTW, I've moved the development issue list to my clone at github:

https://github.com/jspuij/openhab/issues?state=open

This is only for development issues. I'll update the wiki page to reflect this.

Original comment by jwsp...@gmail.com on 2 Oct 2013 at 9:10

[GoogleCodeExporter]

I'll provide binaries for my 1.4.0-zwave branch using cloudbees from now on:

https://jspuij.ci.cloudbees.com/job/openhab-zwave/

Will add this to the wiki as well

Original comment by jwsp...@gmail.com on 2 Oct 2013 at 9:26

[GoogleCodeExporter]

If I try to connect to cloudbees the newly created github authorization fails.

Original comment by DerRo...@gmail.com on 2 Oct 2013 at 10:54

[GoogleCodeExporter]

Hmmm.. It should be public, without authorization. Hold on.

Original comment by jwsp...@gmail.com on 2 Oct 2013 at 11:05

[GoogleCodeExporter]

I made it publicly accessible and now it's gone... long live cloudbees.

Original comment by jwsp...@gmail.com on 2 Oct 2013 at 11:15

[GoogleCodeExporter]

Ah apparently it takes a little time before everything is processed. It should 
be accessible now.

Original comment by jwsp...@gmail.com on 2 Oct 2013 at 11:18

[GoogleCodeExporter]

Yes it works.

Original comment by DerRo...@gmail.com on 2 Oct 2013 at 11:29

[GoogleCodeExporter]

I have asked for having the quota increased: 
https://code.google.com/p/support/issues/detail?id=31068

Original comment by kai.openhab on 4 Oct 2013 at 6:34

[GoogleCodeExporter]

Ben, All.. I've finished configuration serialization 
https://github.com/jspuij/openhab/issues/5. Starting the binding saves me about 
2 minutes ;-)

Original comment by jwsp...@gmail.com on 4 Oct 2013 at 12:49

[GoogleCodeExporter]

BTW, Thanks Kai...

Original comment by jwsp...@gmail.com on 4 Oct 2013 at 12:49

[GoogleCodeExporter]

Nice one JwS. Is it in your clone? Do I need to delete those existing config 
files I generated for you the other day or is the format unchanged? I will 
pull, compile and test ASAP!

Original comment by ben.jone...@gmail.com on 4 Oct 2013 at 7:26

[GoogleCodeExporter]

Format is the same, so you should be good to go immediately.

Original comment by jwsp...@gmail.com on 4 Oct 2013 at 7:54

[GoogleCodeExporter]

Seems to be working nicely. Great work again! Just need to wait for my door 
sensor to finally initialise and serialise itself, and then the whole network 
should come up instantly on a restart. Noticed you create a sub-directory for 
the binding version #. Probably quite sensible going forward - I presume the 
idea is when I build a new version of the binding I can just copy the previous 
version xml files into the new directory and it will save having to rebuild 
them?

Original comment by ben.jone...@gmail.com on 4 Oct 2013 at 10:50

[GoogleCodeExporter]

Got a quick question JwS - I am seeing a lot of 'noise' in my openHAB event log 
due to all the ZWave info updates. I have cut my info bindings right back to 
just the network stats and the NodeDead flag, but every 10s I still see about 
30 event updates. In order to reduce the frequency of these updates I presume I 
can just increase the 'refresh timeout' in openhab.cfg for the ZWave binding? 
And since I don't have polling turned on for any nodes this won't effect 
anything else? Just wondering if this info polling should have a separate 
config value - since in reality would you ever want this stuff to be updated so 
frequently? I was thinking once every 10mins would be enough - or maybe even an 
hour? Or can I just do this by setting the polling interval for each individual 
'info' item binding?

Not a big issue - just something I thought I would note down while I remembered!

Original comment by ben.jone...@gmail.com on 4 Oct 2013 at 11:05

[GoogleCodeExporter]

Yeah. I was thinking about fixing that. I don't like the current structure with 
the refresh in it and the AbstractActiveBinding inheritance. I was thinking 
about removing it and doing everything with events. For now increase the 
refresh time, or you can add a specific binding refresh time to an info binding 
as well I think...

Original comment by jwsp...@gmail.com on 7 Oct 2013 at 6:47

[GoogleCodeExporter]

JwS - got something a little weird going on - nothing major but I am unable to 
track down what is causing it so thought you might be able to help. Attached is 
a log showing the issue.

If you search for 'ERROR' you should find 5 instances. You can ignore the 
first, that is just my door sensor being asleep. 

The other 4 occur at two instances. Both when I am turning off all my lights - 
the first when I went to bed at 10:59pm and the second at 07:31am when the sun 
rises. I have simple rules for turning off all lights at these times.

And it is always Node 3 - which happens to be the only AeoTec Smart Energy 
Micro Switch device I have.

I have tried manually turning this switch on/off both at the wall, and via 
openHAB, and I can't reproduce the error. It only seems to happen when it is 
doing the 'all lights off' rule.

What causes the controller to cancel a message? Is it the device itself 
bouncing it back? Or is the controller getting overloaded with too many things 
to do at once? Seems a bit odd that it is ALWAYS node 3, but maybe that is just 
because of the order of execution?

Any ideas? It is not a major issue, as everything seems to be working fine 
regardless, I am just keeping an eye on the ZWave stats however and seeing this 
'cancelled' count increasing each day is bugging me ;).

Original comment by ben.jone...@gmail.com on 9 Oct 2013 at 9:37

[GoogleCodeExporter]

PS - I will email you the log since I can't attach it here anymore.

Original comment by ben.jone...@gmail.com on 9 Oct 2013 at 9:37

[GoogleCodeExporter]

Disclaimer: have not looked at the log yet.

It's probably nothing. Cancels (and retransmissions) occur when the
controller both receives a message from our binding, and is also sending a
message from the network to the binding. Although the serial communications
channel would certainly allow for it, the Z-Wave controller is only half
duplex.

Your energy meter will probably report a change in the power consumption
immediately after switching the device on or off... This message is received
by the binding while it it's flushing its queue with "off" messages to all
nodes. For the binding it's not a problem, but the controller cannot process
the next "off" command for another node we send, while its also sending us
the new power level of the previous node. Hence the CAN.

We receive the CAN and resend the canceled message by the controller. I have
exactly the same issue with my Fibaro WALL switch and my all off sequence.
Nothing to worry about and nothing we can/should do about it. You never know
when an unsollicited message from the network arrives. It can always happen
while we send a message.

JwS

Original comment by jwsp...@gmail.com on 10 Oct 2013 at 11:19

[GoogleCodeExporter]

We could lower the error level to have it disappear from the logs, but in case 
there are many it indicates an error in our own transaction handling. We have a 
transaction semaphore that guards that a new message is not sent before the 
previous sequence of messages is finished. This way we can guard that we don't 
create CAN messages ourselves.

This does not protect us however from unsollicited messages from one of the 
nodes in the network.

Original comment by jwsp...@gmail.com on 10 Oct 2013 at 11:33