Zwave js - Vision Security ZD2102 missing entity

Extreme79 commented 3 years ago

The problem

The ZD2102 provide 2 separate inputs: one classic magnetic reed and one internal additional input that can be used to monitor another opening (I use it to monitor the window and the outdoor cover with a single device). Every input is treated as a separate, indipendent thing in the same device. With aeotec zstick gen5+ (as secondary controller) included in Home Assistant with Zwave JS I only see an entity for the main reed, but not for the secondary input. This avoid a full use of the device. Please check it.

What is version of Home Assistant Core has the issue?

core-2021.6.2

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant Supervised

Integration causing the issue

Zwave JS

Link to integration documentation on our website

https://www.home-assistant.io/integrations/zwave_js/

Example YAML snippet

No response

Anything in the logs that might be useful for us?

I checked the node log, after set it to silly. Opening the reed and the secondary input it show lines only for the main reed, no lines on secondary input opening. Nothing to show :(

Additional information

I know that the same issue was happened also in other platforms. I found an old thread on openhab forum that seems talking about the same issues in the various posts. https://community.openhab.org/t/solved-vision-door-windows-sensor-zd2102-open-close-and-tamper-are-not-triggered-to-different-events/27972/4

kpine commented 3 years ago

Please attach a dump of the node of concern.

Extreme79 commented 3 years ago

@kpine you were right.. I forgot it. ZD2102 node ids: 7, 8, 9, 10, 11, 14, 51 (where the missing internal contact is used in all the devices except node id 14). I also add attached log lines captured for node id 7, opening main reed, opening the reed connected to external switch then closing it, closing main reed.

Here's also a lack of configuration parameters, essentially the parameter that enable/disable the external switch and wakeup time (that could be set between 10min >> 1 week). But here's also a lack of documentation, I think. enabling/disabling external contact should be the parameter 1, and for wakeup I read that is needed to set it using COMMAND_CLASS_WAKE_UP (unfortunately I'm not too skilled on zwave to fully understand that, I'm learning :) )

I'm here for other infos you need.

zwave_js_dump.zip node7_log.txt

kpine commented 3 years ago

First thing, If you want the software to support the configuration parameter, you need to go to node-zwave-js and submit a request to have it added (or do it yourself). The parameter will not be available in HA until the device config specifies it. If you use zwavejs2mqtt, it has the capability to set custom config parameter values, so you can change it w/o the device config file being correct, but this isn't supported in HA.

Continuing on with the entities issue, according to your dump, you have these values:

A Binary Sensor value. I presume this matches a binary_sensor entity which reflects the reed switch operation.
Home Security - Sensor Status, which is also the reed switch notification.
Home Security - Cover status, which is the tamper switch notification
Home Security - Unknown, which is the external switch notification

The problem with all of these Home Security values is that they have the metadata type of "any", which is not something HA supports (and in my opinion it is incorrect for zwave-js to reports this type). This is also seen in issue #46982 (another Vision device...). For example:

              {
                "endpoint": 0,
                "commandClass": 113,
                "commandClassName": "Notification",
                "property": "Home Security",
                "propertyKey": "unknown",
                "propertyName": "Home Security",
                "propertyKeyName": "unknown",
                "ccVersion": 2,
                "metadata": {
                  "type": "any",
                  "readable": true,
                  "writeable": true
                },
                "value": 254
              },

So until this metadata type gets corrected to "number", HA cannot support them. That needs to be solved in node-zwave-js first.

However, I'm not sure that helps you. Let's say those values were supported as sensors today, the next problem is that they will never idle, unless zwave-js does an auto-idle based on time. That makes them pretty useless. The device itself never idles those values, so you can never tell when the switches go from off <=> on. For this you need to use the Alarm V1 types, alarm level and alarm type. Z-Wave JS is not exposing these values because they are Alarm V1 values, but the device supports V2 notifications.

Even if you have the V1 values exposed, I think there are problems. IMO, this device incorrectly uses the notification system to report the external sensor events. Let's look at the documentation posted from the OH thread.

Note that Alarm Type and Alarm Level are Alarm (Notification) V1 CC values. The Alarm Event value is from Notification (Alarm) V2 CC. In order to distinguish events from any of the switch types, you are required to examine both V1 values plus the V2 Alarm Event value. If you use only the V1 values, you cannot distinguish any event at all: if alarm type is 7 and alarm level is 0xFF, which switch is it? If you look at only the V2 value, you never know if the value was on or off. This does not seem like behavior that falls within the Z-Wave spec to me, but maybe I'm wrong. Z-Wave JS does not support it anyways.

Here's some analysis of the raw messages coming from the device to illustrate more.

Here's the external switch sending an "open" notification.

2021-06-06T13:16:14.879Z SERIAL « 0x0112000400070a710507ff00ff07fe0000b500db                          (20 bytes)
2021-06-06T13:16:14.882Z CNTRLR   [Node 007] [~] [Notification] notificationMode: "pull" [Endpoint 0] [internal]
                                   => "pull"
2021-06-06T13:16:14.884Z SERIAL » [ACK]                                                                   (0x06)
2021-06-06T13:16:14.898Z DRIVER « [Node 007] [REQ] [ApplicationCommand]
                                  +-[NotificationCCReport]
                                      notification type:   7
                                      notification status: 255
                                      notification event:  0xfe
2021-06-06T13:16:14.907Z CNTRLR   [Node 007] [~] [Notification] Home Security[unknown]: 254 => 254  [Endpoint 0]

The raw data received from the device is the first line. It can be broken down as:

0x0112000400070a710507ff00ff07fe0000b500db
  0112000400070a
    71  notification cc
    05  alarm report command
    07  alarm type - 7
    ff  alarm level - 0xff (255)
    00  zensor net source node id
    ff  z-wave alarm status - alarm enabled
    07  z-wave alarm type - home security
    fe  z-wave alarm event - unknown event/state (external sensor)
    00  number of event parameters - 0
  00b500db

The useful values here are the V1 alarm type (7) and alarm level (0xff) and z-wave alarm event (0xfe). Those values match the manual that was posted, indicating the external sensor is open. But if you look at V1 values alone, you don't know it's the external sensor.

Prior that that, there was also the notification for the reed sensor opening:

2021-06-06T13:16:11.656Z SERIAL « 0x0112000400070a710507ff00ff07020000b50027                          (20 bytes)
2021-06-06T13:16:11.658Z CNTRLR   [Node 007] [~] [Notification] notificationMode: "pull" [Endpoint 0] [internal]
                                   => "pull"
2021-06-06T13:16:11.659Z SERIAL » [ACK]                                                                   (0x06)
2021-06-06T13:16:11.661Z DRIVER « [Node 007] [REQ] [ApplicationCommand]
                                  +-[NotificationCCReport]
                                      notification type:   Home Security
                                      notification status: 255
                                      notification state:  Intrusion
2021-06-06T13:16:11.665Z CNTRLR   [Node 007] [~] [Notification] Home Security[Sensor status]: 2 =>  [Endpoint 0]
                                  2

Raw data:

0x0112000400070a710507ff00ff07020000b50027
  0112000400070a
    71  notification cc
    05  alarm report command
    07  alarm type - 7
    ff  alarm level - 0xff (255)
    00  zensor net source node id
    ff  z-wave alarm status - alarm enabled
    07  z-wave alarm type - home security
    02  z-wave alarm event - intrusion (reed switch)
    00
  00b50027

Again, notice the only difference here is the V2 z-wave alarm event. The alarm type (7) and alarm level (0xff) are both the same as the external switch. It's impossible to distinguish the notification meaning based on the V1 types alone. Thus you need to examine all 3 to distinguish the event type.

The tamper switch itself never goes idle, you cannot tell if it's open or close as it always sends the same alarm level (0xFF). At best you know something happened.

So in HA, there's nothing that can be done to support the tamper and external sensor unless node-zwave-js presents a different metadata type than "any".

Even if the event metadata type was correct, or V1 alarm values were exposed, it's unclear that this device would still be usable. I would advise starting a discussion topic, or submitting in issue to see if there's any way for the driver to support this. The way I see it, you could try to argue that the Alarm V1 values (alarm level and alarm type) should be exposed, but those two values alone are not enough to know which sensor is actually activated. You need that third alarm event value, and there's no way to tie all three together into a single event.

Extreme79 commented 3 years ago

Your explanation it's very clear. Thanks for your time. I'll try to open an issue in Zwave js too, linking this issue.

My main controller is actually a zipato zipabox (I'm playing with aeotec zstick as secondary controller/ HA / ZwaveJS to abandon the old zipabox), where this sensor is working fine since years. So it's technically possible to distinguish switches and as I understand the only way is a fix from zwavejs to avoid the "any" type in metadata.

Extreme79 commented 3 years ago

after the new zwavejs release, something changed. New entities was detected by HA, but the issue isn't solved. New detected entities never change their values, and nothing seems to be related to the external switch. I attach the silly log that I also attached on the closed issue on zwavejs github. silly_log_ZD2102_zwaveJS_latest.txt

The sequence is: open main door, open second door (external switch) an then close it, close main door, open device cover and then close it.

here's the list of entities that HA detect after re-interview here the list of entities that now HA create after re-interview with new zwave-js release ("porta ingresso" is the node 7 device name):

sensor.porta_ingresso_home_security_sensorstatus (value "Intrusion" and it never change)_
binary_sensor.porta_ingresso_home_securityintrusion (value "Not Safe" and never change)_
sensor.porta_ingressobasic (value 0.0 and never change)_
binary_sensor.porta_ingressoany (value on/off based on main reed status, it works)_
sensor.porta_ingresso_batterylevel (value is the battery percentage, it works)_
binary_sensor.porta_ingresso_low_batterylevel (value actually "Normale" I suppose it works)_
sensor.porta_ingresso_home_security_coverstatus (actual value "Tampering, product cover removed", do not works, the device is normally installed and closed. It never change on cover opening/close)_
binary_sensor.porta_ingresso_home_securitytamper (actual value "Not Safe" and it never change)_
sensor.porta_ingresso_home_securityunknown (actual value 254 and it never change)_

kpine commented 3 years ago

We would need to see a new network dump.

kpine commented 3 years ago

Even without the dump (which I'd still like to look at), I can quote my original comment to explain the reason why the sensor values never change:

However, I'm not sure that helps you. Let's say those values were supported as sensors today, the next problem is that they will never idle, unless zwave-js does an auto-idle based on time. That makes them pretty useless. The device itself never idles those values, so you can never tell when the switches go from off <=> on. For this you need to use the Alarm V1 types, alarm level and alarm type. Z-Wave JS is not exposing these values because they are Alarm V1 values, but the device supports V2 notifications.

Those sensors don't change because the device never reports any other values.

kpine commented 3 years ago

While we're waiting for that network dump, do you have any new disabled entities? I would expect two sensor entities, alarm_level and alarm_type, and they would be disabled by default.

Extreme79 commented 3 years ago

@kpine I attached the new zwave network dump and attached the latest log from the device after the zwavejs update (that I also attached on zwavejs closed issue). I had disabled entities on the device, but I enabled all to avoid any mistake.

About the data update from the device: as I said I'm not sufficently skilled on zwave protocol so I risk to do not fully understand some log details, but talking to a lot higher level, the main zwave controller (zipato zipabox) is able to distinguish the two zd2102 inputs, and also the tamper, because opening the device cover cause the trigger of tamper alarm on the controller.. so I can expect that the device is able to send somenthing on the network about these events. :-?

zwave_js_dump_10_06_2021.zip silly_log_ZD2102_zwaveJS_latest.txt

kpine commented 3 years ago

@Extreme79 Looks like the V1 alarm values didn't get detected. Would you be able to re-interview the device and capture the debug logs for that?

the main zwave controller (zipato zipabox) is able to distinguish the two zd2102 inputs, and also the tamper, because opening the device cover cause the trigger of tamper alarm on the controller.. so I can expect that the device is able to send somenthing on the network about these events. :-?

I can only assume the Zipato must have specialized code to deal with this device, that's pretty common for commercial closed-source hubs. Z-Wave JS and HA try to work within the Z-Wave specifications, and avoid any custom device code as much as possible. It is impossible to support the functionality of this device without special code because it doesn't stay strict to the standards.

Regarding the tamper alarm, I don't know what Zipato does, but it's impossible to determine whether the cover is open or closed based on the alarm message alone. The device sends the same message whether you are opening or closing it. You can track your own state (in something like an input_boolean), but it's possible for it to be out of sync with the real state.

If you setup an automation with a state trigger for the tamper sensor, it will trigger even if the value doesn't change, simply omit a to or from state in the trigger.

Extreme79 commented 3 years ago

@Extreme79 Looks like the V1 alarm values didn't get detected. Would you be able to re-interview the device and capture the debug logs for that?

Sure, I re-interviewed it few minutes ago and here, attached the result. Opened cover started the interview closed the cover after some seconds interview finished (it happen when I close the cover) zd2102_new_interview_silly.txt

the main zwave controller (zipato zipabox) is able to distinguish the two zd2102 inputs, and also the tamper, because opening the device cover cause the trigger of tamper alarm on the controller.. so I can expect that the device is able to send somenthing on the network about these events. :-?

I can only assume the Zipato must have specialized code to deal with this device, that's pretty common for commercial closed-source hubs. Z-Wave JS and HA try to work within the Z-Wave specifications, and avoid any custom device code as much as possible. It is impossible to support the functionality of this device without special code because it doesn't stay strict to the standards.

I'll try to ask to vision security about that and I'll ask for an updated firmware, if exists.

Regarding the tamper alarm, I don't know what Zipato does, but it's impossible to determine whether the cover is open or closed based on the alarm message alone. The device sends the same message whether you are opening or closing it. You can track your own state (in something like an input_boolean), but it's possible for it to be out of sync with the real state.

If you setup an automation with a state trigger for the tamper sensor, it will trigger even if the value doesn't change, simply omit a to or from state in the trigger.

this sound me partially familiar. I noticed that sometimes the controller report a wrong device state (say open when closed) for the main or external contact (but more often for the external) and I need to open and close manually to re-align the state to the real world. Following another log I took after the new interview.

started from main reed open
I opened the external switch
I closed the external switch
I closed the main reed. another_silly_log_zd2102.txt

Extreme79 commented 3 years ago

Hi. I updated to 0.1.26 that bumps to zwavejs 7.7.4. I reinterviewed the device after a reboot of zwavejs integration and ha server. No changes on the device entities when I open the external switch BUT I see something different in the log when I open the external switch: an alarmLevel that change from 0 to 255 on external switch open, and the same alarmLevel that change back to 0 on external switch close. Now seems that zwavejs do it's dirty job (or job on dirty device, see it as you want :) ) but something seems to lacks on HA side. Here is the latest log starting from all closed (tamper, main reed, external switch):

main reed open
external switch open
external switch close
main reed close

silly_zd2102_integration0126_zwave774.txt

(sorry I closed the issue as an error, next I reopened it)

kpine commented 3 years ago

You can create a template sensor to do whatever you want using those values.

Extreme79 commented 3 years ago

You can create a template sensor to do whatever you want using those values.

Sure.. having the entity, But the the entities created by home assistant stay unchanged after the new interview, and looks like before and no changes are shown on entities when external switch is opened. So I can't templating. There's a lack of the entity rapresenting the value that in the log now is changing with opened external switch. (Or I'm not understanding something)

the key is here, there's no entity that reflect this:

2021-06-15T23:51:00.614Z CNTRLR   [Node 007] [~] [Notification] alarmType: 7 => 7                   [Endpoint 0]
2021-06-15T23:51:00.618Z CNTRLR   [Node 007] [~] [Notification] alarmLevel: 255 => 0               [Endpoint 0]
2021-06-15T23:51:00.620Z SERIAL » [ACK]                                                                   (0x06)
2021-06-15T23:51:00.624Z DRIVER « [Node 007] [REQ] [ApplicationCommand]
                                  └─[NotificationCCReport]
                                      V1 alarm type:  7
                                      V1 alarm level: 0

kpine commented 3 years ago

There should be alarm type and alarm level entities that are disabled by default. Feel free to attach another dump file if you don't see them.

kpine commented 3 years ago

I'd also suggest enabling debug logging while you toggle the switches to see what is happening when the notifications are received by HA. Depending on the order the messages are received, it may be very difficult to detect the events reliably. You can use logger.set_level service to temporarily enable debug logging.

service: logger.set_level
data:
  zwave_js_server: debug
  homeassistant.components.zwave_js: debug

Set to info when done to return to normal log levels.

Extreme79 commented 3 years ago

There should be alarm type and alarm level entities that are disabled by default. Feel free to attach another dump file if you don't see them.

Here my new zwave dump I enabled all the disabled entities as first thing before opening the issue: actually I have no disabled zwave_js_dump_18_06_2021.zip

Tomorrow I'll enable logging to explore deeply.

kpine commented 3 years ago

Sorry, but it's the same problem as before. The V1 values are now created, but again with the wrong metadata type. The type should be number, and I know in the past these values used to be created in that fashion. @AlCalzone is this expected?

              {
                "endpoint": 0,
                "commandClass": 113,
                "commandClassName": "Notification",
                "property": "alarmType",
                "propertyName": "alarmType",
                "ccVersion": 2,
                "metadata": {
                  "type": "any",
                  "readable": true,
                  "writeable": true
                },
                "value": 7
              },
              {
                "endpoint": 0,
                "commandClass": 113,
                "commandClassName": "Notification",
                "property": "alarmLevel",
                "propertyName": "alarmLevel",
                "ccVersion": 2,
                "metadata": {
                  "type": "any",
                  "readable": true,
                  "writeable": true
                },
                "value": 0
              },

AlCalzone commented 3 years ago

Probably also because the device claims it doesn't support them: grafik

AlCalzone commented 3 years ago

PR is up

Extreme79 commented 3 years ago

After my holidays, I come back to play with home assistant. After the latest zwave-js released implementations, all apperently gone ok with my main door vision sensor. So I re-interviewed another vision zvwave.. and I got different entities.. :-? So I decided to remove integration, removing all entities and re-start from the scratch. With lates HA release (2017.7.2) and latest zwave-js something goes different: I no more have the alarmLevel entity! so no more detection of secondary switch. This is a little bit strange because, before re-starting from the beginning, where alarm entity was present it worked as expected. As soon I reconfigured the zwave network I'll attach the dump and logs. EDIT: after some reboots (of zwave integration too) the alarmLevel appeared. I have to reinterview the others zd2102, then I'll reboot ha & ZW and check.

github-actions[bot] commented 3 years ago

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍 This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

home-assistant / core