home-assistant / core

:house_with_garden: Open source home automation that puts local control and privacy first.
https://www.home-assistant.io
Apache License 2.0
74.15k stars 31.12k forks source link

Vision Security ZG8101 sensor showing alarm type and level but no values #46982

Closed macf0x closed 3 years ago

macf0x commented 3 years ago

The problem

Vision Security ZG8101 (Garage Door Tilt Sensor) is not presenting all entities in HA. Values for alarm type and level are populate in the Node view of Z-wave-js (notifications/113), but the values aren't shown in HA. Also these initial entities are hidden in HA, when I exclude/include.

These entities never receive values 23-113-0-alarmType 23-113-0-alarmLevel

These appear to be duplicates of the above entities and do have values, but do not appear as addressable entities. 23-113-0-Home Security-Cover status 23-113-0-Home Security-Sensor status 23-113-0-Home Security-unknown

Even when unhiding the entities, the state is never updated. I believe it maybe related to the these values being classed as notifications (113), rather than Binary Sensor(48) or Basic(32).

I have opened an issue with zwave-js2mqtt, but advised to open one here as the no issue was found. https://github.com/zwave-js/zwavejs2mqtt/issues/592

Happy to provide more detail if below is insufficient.

Please see the attached Logs and Entity registry snippet. If you need further details please let me know.

Z-Wave JS to MQTT Current version: 0.4.4 MQTT Gateway disabled Using Mosquitto broker Current version: 5.1

What is version of Home Assistant Core has the issue?

core-2021.2.3

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

zwave_js

Link to integration documentation on our website

https://www.home-assistant.io/integrations/zwave_js/

Example YAML snippet

# Put your YAML below this line

Anything in the logs that might be useful for us?

# Put your logs below this line
15:59:22.447 CNTRLR [Node 023] treating BasicCC Set as a report
2021-02-24 15:59:22.450 INFO ZWAVE: Node 23: value updated: 48-0-Any false => true
2021-02-24 15:59:22.508 INFO ZWAVE: Node 23: value updated: 113-0-Home Security-Sensor status 2 => 2
2021-02-24 15:59:23.046 INFO ZWAVE: Node 23: value updated: 113-0-Home Security-unknown 254 => 254
2021-02-24 15:59:26.805 INFO ZWAVE: Node 23: value updated: 113-0-Home Security-unknown 254 => 254
15:59:28.977 CNTRLR [Node 023] treating BasicCC Set as a report
2021-02-24 15:59:28.979 INFO ZWAVE: Node 23: value updated: 48-0-Any true => false
2021-02-24 15:59:29.082 INFO ZWAVE: Node 23: value updated: 113-0-Home Security-Sensor status 2 => 2
15:59:29.680 CNTRLR [Node 023] treating BasicCC Set as a report
2021-02-24 15:59:29.681 INFO ZWAVE: Node 23: value updated: 48-0-Any false => true
2021-02-24 15:59:29.699 INFO ZWAVE: Node 23: value updated: 113-0-Home Security-Sensor status 2 => 2
2021-02-24 15:59:34.302 INFO ZWAVE: Node 23: value updated: 113-0-Home Security-Cover status 0 => 3
2021-02-24 15:59:36.698 INFO ZWAVE: Node 23: value updated: 113-0-Home Security-Cover status 3 => 3
15:59:38.717 CNTRLR « [Node 023] received wakeup notification
2021-02-24 15:59:38.719 INFO ZWAVE: Node 23 is now awake
15:59:38.720 CNTRLR [Node 023] The node is now awake.
2021-02-24 15:59:38.770 INFO ZWAVE: Node 23: value updated: 128-0-level 83 => 83
2021-02-24 15:59:38.771 INFO ZWAVE: Node 23: value updated: 128-0-isLow false => false
15:59:38.774 CNTRLR « [Node 023] received response for battery information:
level: 83
15:59:38.781 CNTRLR [Node 023] WakeUpCC: doing a partial interview...
15:59:38.782 CNTRLR » [Node 023] retrieving wakeup interval from the device...
2021-02-24 15:59:38.836 INFO ZWAVE: Node 23: value updated: 132-0-wakeUpInterval 7200 => 7200
2021-02-24 15:59:38.837 INFO ZWAVE: Node 23: value updated: 132-0-controllerNodeId 1 => 1
15:59:38.841 CNTRLR « [Node 023] received wakeup configuration:
wakeup interval: 7200 seconds
controller node: 1
15:59:38.842 CNTRLR [Node 023] AssociationCC: doing a partial interview...
15:59:38.843 CNTRLR » [Node 023] querying association group #1...
15:59:38.893 CNTRLR « [Node 023] received information for association group #1:
maximum # of nodes: 5
currently assigned nodes: 1
15:59:38.895 CNTRLR [Node 023] BasicCC: doing a partial interview...
15:59:38.896 CNTRLR » [Node 023] querying Basic CC state...
15:59:38.948 CNTRLR « [Node 023] received Basic CC state:
current value: 99
15:59:38.959 CNTRLR [Node 023] BinarySensorCC: doing a partial interview...
15:59:38.960 CNTRLR » [Node 023] querying current value...
2021-02-24 15:59:39.007 INFO ZWAVE: Node 23: metadata updated: 48-0-Any
2021-02-24 15:59:39.009 INFO ZWAVE: Node 23: value updated: 48-0-Any true => true
15:59:39.012 CNTRLR « [Node 023] received current value: true
15:59:39.014 CNTRLR [Node 023] NotificationCC: doing a partial interview...
15:59:39.015 CNTRLR [Node 023] Interview stage completed: CommandClasses
15:59:39.015 CNTRLR [Node 023] Interview stage completed: OverwriteConfig
15:59:39.016 CNTRLR » [Node 023] requesting node neighbors...
15:59:39.065 CNTRLR « [Node 023] node neighbors received: 1, 17
15:59:39.070 CNTRLR [Node 023] Interview stage completed: Neighbors
15:59:39.070 CNTRLR [Node 023] Interview completed
2021-02-24 15:59:39.074 INFO ZWAVE: Node 23: interview completed, all values are updated
15:59:40.076 CNTRLR » [Node 023] Sending node back to sleep...
15:59:51.308 CNTRLR [Node 023] The node did not respond after 1 attempts.
It is probably asleep, moving its messages to the wakeup queue.
2021-02-24 15:59:51.318 INFO ZWAVE: Node 23 is now asleep
15:59:51.320 CNTRLR [Node 023] The node is now asleep.

Entity config 265-2562-8202 (Node23).txt

Zwave-js log showing Basic CC providing value. zwavejs.log

probot-home-assistant[bot] commented 3 years ago

Hey there @home-assistant/z-wave, mind taking a look at this issue as its been labeled with an integration (zwave_js) you are listed as a codeowner for? Thanks! (message by CodeOwnersMention)

AlCalzone commented 3 years ago

To give some additional input: I'm not sure where alarmLevel and alarmType come from - maybe from the initial interview. The device communicates using BasicSet (which gets mapped to a binary sensor) and notifications values for tamper and intrusion detection.

marcelveldt commented 3 years ago

Please create a dump and share it here so we can have a look at all values.

Home Assistant --> Settings --> Integrations --> Z-Wave JS --> Configure --> Create dump

Thanks!

kpine commented 3 years ago

I have the same "issue" with the ZCOMBO-G. It supports Notification V8 but zwave-js generates v1 values for it, and so HA generates disabled entities for them.

zwave-js tells HA that there are alarmLevel and alarmType values. HA has know way of knowing that they aren't ever functional. At least they are disabled by default, you have to really go out of your way to do this.

I have a ZG8101 on OZW still. There are no v1 values in HA for this device. It supports Basic Set and Notification CC V2, and OZW also maps Basic Set to Binary Sensor. When the door opens and closes, both Basic Set and Notification Reports are set.

For comparison with OZW (sorry!), OZW only creates v1 alarm values for these scenarios:

My interpretation is that a device can advertise it supports V1 alarms, but that is in addition to the V2+ alarms ("the device implements Notification CC V2 Notification Types as well as proprietary Alarm CC V1 Alarm Types and Alarm Levels"). Therefore it cannot support a V2+ command class and only send V1 alarms if that bit is set. I always assumed it's for backwards compatibility with controllers that only supported V1. But the Notification CC is complicated, so ... 🤷

These values should be disabled in zwave-js if this is really a problem.

AlCalzone commented 3 years ago

These values should be disabled in zwave-js if this is really a problem.

I agree but I need to understand why they even show up. That requires a log with a complete interview of the device.

kpine commented 3 years ago

OK, the title of this issue misled me a bit and I didn't look closely enough. There are a few problems:

  1. zwave-js creates v1 alarm values, which aren't functional. The device even reports during the interview that it doesn't support V1 alarms.
  2. HA does not generate entities for the "Home Security" sensors. This is a discovery issue as HA expects notification CC values to be type "number", but these values are type "any".
  3. The sensor values that are created are permanently set. As far as I can see, the alarm status is the same whether the alarm was activated, e.g. for cover open and cover close alarms both report status 0xff. Is this a device bug, or does V2 work this way? zwave-js does not auto-reset the value to idle on its own.

OZW in comparison creates values that are the same type as the standard notification types. It automatically auto-idles values after 5 seconds for V2 and V3.

kpine commented 3 years ago

These values should be disabled in zwave-js if this is really a problem.

I agree but I need to understand why they even show up. That requires a log with a complete interview of the device.

Here you go (node 1 is the Vision): zwavejs_1.log

AlCalzone commented 3 years ago

You mean node 8, right?

AlCalzone commented 3 years ago

My guess is: The alarm values are defined with static metadata, which is why HA sees them, although they are never populated. I should just create the metadata dynamically.

kpine commented 3 years ago

You mean node 8, right?

Uh, yeah that one... :tired_face:

kpine commented 3 years ago

That leaves the question of what to do in HA with notification values like this then:

                        {
                            "endpoint": 0,
                            "commandClass": 113,
                            "commandClassName": "Notification",
                            "property": "Home Security",
                            "propertyKey": "Sensor status",
                            "propertyName": "Home Security",
                            "propertyKeyName": "Sensor status",
                            "ccVersion": 2,
                            "metadata": {
                                "type": "any",
                                "readable": true,
                                "writeable": true
                            },
                            "value": 2
                        },
                        {
                            "endpoint": 0,
                            "commandClass": 113,
                            "commandClassName": "Notification",
                            "property": "Home Security",
                            "propertyKey": "Cover status",
                            "propertyName": "Home Security",
                            "propertyKeyName": "Cover status",
                            "ccVersion": 2,
                            "metadata": {
                                "type": "any",
                                "readable": true,
                                "writeable": true
                            },
                            "value": 3
                        },

@AlCalzone is there a reason these values are "any" instead of "number"? I notice the device reports "unknown" for the notification type, intead of push or pull. Does that effect it? The V2 class still supports getting the list of supported notifications. It looks like you only ask for V3 and up.

Another problem, at least for these devices, is that they never change so they are pretty useless.

AlCalzone commented 3 years ago

The V2 class still supports getting the list of supported notifications. It looks like you only ask for V3 and up.

V2 supports requesting the supported notification types. V3+ supports querying each type's supported events. This however implies that we only create the metadata for V3+ devices (which is why you have the default Any metadata). It could be done just-in-time for the V2 devices though - HA just won't know ahead of time which values exist.

AlCalzone commented 3 years ago

😑 I just figured out after about half a year why notification mode is always unknown.

kpine commented 3 years ago

V2 supports requesting the supported notification types. V3+ supports querying each type's supported events.

Ah right, I confused the loops, it's checked at the top. Thanks.

kpine commented 3 years ago

😑 I just figured out after about half a year why notification mode is always unknown.

Better late than never! 😀

AlCalzone commented 3 years ago

@kpine do you have a way to test the current zwave-js master branch (regarding notificationMode and the alarm values)? You'll need to re-interview the device for that to take effect.

kpine commented 3 years ago

@AlCalzone Here's a log: zwavejs_1.log

The first time I restarted zwavejs2mqtt it wanted to re-interview the node. Not sure if that was a problem from the previous shutdown, or if it was expected to. After I woke up the node to have it re-interviewed, it still showed the v1 alarm values loaded from cache. I didn't try the actual re-interview command, would that would have removed them? Anyways, instead I deleted the cache and started over, which is the log above. No more alarm values.

AlCalzone commented 3 years ago

I didn't try the actual re-interview command, would that would have removed them?

Yes, it would have. Just for this one node, instead of all of them.

No more alarm values.

👍🏻

macf0x commented 3 years ago

Sorry for the delay.

Update to; Core 2021-3.2 Z-Wave JS to MQTT 0.6.0 Z-Wave JS 0.1.10

Excluded and included

No longer seeing the the alarm/type entities at all now with the latest release. Can these be forced?

Looking at raw ZWJS logs indicates the values are interviewed and reported, but the value never returns to 0 as was the case with OZW, thus you can't trigger and automation.

I'm actually using this on a letterbox, which has a door to put mail in and a separate door to retrieve mail.

Log of Tamper/Alarm zwavelog node 24.txt

HA registry Zwave Node 24.txt

node_24.json.txt

zwave_js_dump.json.txt

kpine commented 3 years ago

No longer seeing the the alarm/type entities at all now with the latest release. Can these be forced?

The fix in node-zwave-js was to remove these values. They aren't supported by the device.

Looking at raw ZWJS logs indicates the values are interviewed and reported, but the value never returns to 0 as was the case with OZW, thus you can't trigger and automation.

That would have to be a question for node-zwave-js, as to whether the v2 notifications should be idled or not. How were you using the values in an automation with OZW? If you open the door, the event triggers. If you close the door it triggers. You don't actually know whether the door is open or not.

Is there a reason you can't use the binary sensor?

AlCalzone commented 3 years ago

Which value ID is not returning to idle? If the notification definition allows a value to be idle, that happens automatically after 5 minutes with no changes, unless the node supports V8, which mandates the idle reset to come from the device.

macf0x commented 3 years ago

Is there a reason you can't use the binary sensor?

I do use the Binary Sensor. But the mailbox has two doors. Originally I was using the notification to trigger mail had been dropped in the top (door). I then use the Binary sensor to detect when the rear door was opened and closed to reset the letterbox to empty. Presumably under OZW it reset the state back after x amount of time.

It's not clear if this is supported under ZWJS?

Which value ID is not returning to idle?

Based on the logs both the Alarm State and Alarm Type

2021-03-07 14:30:00.940 INFO ZWAVE: Node 24: value updated: 113-0-Home Security-unknown 254 => 254 14:30:05.708 SERIAL « 0x0110000400180a7105070000ff07fe00008c (18 bytes)

2021-03-07 14:30:11.399 INFO ZWAVE: Node 24: value updated: 113-0-Home Security-Sensor status 2 => 2 14:30:12.476 SERIAL « 0x010900040018032001ff37 (11 bytes)

Do you need further details from the device to determine if it supports returning to idle?

AlCalzone commented 3 years ago

Based on the logs both the Alarm State and Alarm Type

2021-03-07 14:30:00.940 INFO ZWAVE: Node 24: value updated: 113-0-Home Security-unknown 254 => 254 14:30:05.708 SERIAL « 0x0110000400180a7105070000ff07fe00008c (18 bytes)

2021-03-07 14:30:11.399 INFO ZWAVE: Node 24: value updated: 113-0-Home Security-Sensor status 2 => 2 14:30:12.476 SERIAL « 0x010900040018032001ff37 (11 bytes)

Where do you read "alarm" here? The latter is the intrusion sensor which should automatically be returned to idle by zwave-js after 5 minutes.

Unless a device supports Notification CC V8, in which case the specs mandate that the device MUST handle it:

Version 8 increases the requirement level for the use of the “State idle” Notification from OPTIONAL to REQUIRED. Supporting nodes implementing version 8 or newer of the Notification Command Class MUST issue a “State idle” Notification when a state variable returns to idle. For instance, the {Smoke alarm::Smoke Detected} Notification MUST be followed by a {Smoke alarm::State idle(Smoke detected)} Notification when the smoke is no longer detected by the sensor. (refer to Figure 16).

From your dump it does look like the device is V2. Does it really never reset?

macf0x commented 3 years ago

"Alarm" is what it was showing under OZW.

Here is the config snippet from OZW. Perhaps that will make more sense.

OZWcfg.txt

From your dump it does look like the device is V2. Does it really never reset?

To be honest I've not seen it return to zero. Mainly because the initial issue was these values were not updating in HA like they were previous under OZW. If that is the expected behaviour I will run some further tests and report back with some ZWJS logs.

At the moment how would one get these notification events in HA?

AlCalzone commented 3 years ago

At the moment how would one get these notification events in HA?

These should be normal values like most other CCs. I don't know if HA has any special handling for Notification CC values in place

"Alarm" is what it was showing under OZW.

Alarm CC was renamed to Notification CC when the V2 specifications for that CC came out. The legacy V1 values that can still be observed are alarmType and alarmLevel, but both are part of the Notification CC and both are only visible if the device supports V1 alarms.

If you really don't see them return to idle, there might be a bug. I'll need zwave-js logs (loglevel debug) to verify if it is a bug in zwave-js or somewhere else. Unfortunately my only devices that I can reliably trigger notifications on do return to idle on their own.

macf0x commented 3 years ago

Updated to 0.6.1

I removed the sensor from the letterbox for more convenient testing. Trigger the notifications. Attached is the debug log and the final state of Node 24

I'm not sure if I should see something in the log idling the values back, but I triggered both, and both values remained at the notification state.

node_24.json.txt zwavejs_351.log

AlCalzone commented 3 years ago

Yeah, that doesn't look like something is getting idled.

kpine commented 3 years ago

I do use the Binary Sensor. But the mailbox has two doors. Originally I was using the notification to trigger mail had been dropped in the top (door). I then use the Binary sensor to detect when the rear door was opened and closed to reset the letterbox to empty. Presumably under OZW it reset the state back after x amount of time.

Not sure I understand this 100%. Are you using the tilt sensor (binary sensor) for the door opening, and the cover (tamper) sensor to detect mail being dropped in?

These should be normal values like most other CCs. I don't know if HA has any special handling for Notification CC values in place

As I mentioned above, HA does not discover these sensors because the type is "any". A new discovery schema would need to be implemented.

2021-03-07 14:30:00.940 INFO ZWAVE: Node 24: value updated: 113-0-Home Security-unknown 254 => 254 14:30:05.708 SERIAL « 0x0110000400180a7105070000ff07fe00008c (18 bytes)

2021-03-07 14:30:11.399 INFO ZWAVE: Node 24: value updated: 113-0-Home Security-Sensor status 2 => 2 14:30:12.476 SERIAL « 0x010900040018032001ff37 (11 bytes)

@AlCalzone I've noticed that I no longer have a "Cover status" entity any longer. I suppose that means zwave-js has never seen a notification with that type yet? I haven't removed the cover since including it, although the act of including the node uses the tamper button, so after including just putting the cover back on should have triggered an event, so I'm not sure why it's missing.

FYI, regarding the return to idle issue, OZW defaults to clearing the status after 5 seconds. As for these particular devices, they never reset, nor do the report an idle value. "Cover status" (status 0xff) is emitted every time the cover is removed or installed. "Sensor status" (0xff) is emitted every time the tilt sensor changes position.

I'm looking through my logs, is the idle time actually 5 hours instead of minutes?

14:30:18.371 SERIAL « 0x011300a80001170a7105070000ff0702000000b968                        (21 bytes)
14:30:18.372 SERIAL » [ACK]                                                                   (0x06)
14:30:18.374 DRIVER « [Node 023] [REQ] [BridgeApplicationCommand]
                      └─[NotificationCCReport]
                          notification type:   Home Security
                          notification status: 255
                          notification state:  Intrusion
14:30:18.376 CNTRLR   [Node 023] [~] [Notification] Home Security[Sensor status]: 2 =>  [Endpoint 0]
                      2
...
19:29:30.806 SERIAL « 0x011300a80001120a7105060000ff0617000000c504                        (21 bytes)
19:29:30.807 SERIAL » [ACK]                                                                   (0x06)
19:29:30.808 DRIVER « [Node 018] [REQ] [BridgeApplicationCommand]
                      └─[NotificationCCReport]
                          notification type:   Access Control
                          notification status: 255
                          notification state:  Window/door is closed
19:29:30.811 CNTRLR   [Node 018] [~] [Notification] Access Control[Door state]: 23 => 2 [Endpoint 0]
                      3
19:30:18.384 CNTRLR   [Node 023] [~] [Notification] Home Security[Sensor status]: 2 =>  [Endpoint 0]
                      0
19:30:35.632 SERIAL « 0x010c00a8000112032001ff00c550                                      (14 bytes)
19:30:35.633 SERIAL » [ACK]                                                                   (0x06)
19:30:35.634 DRIVER « [Node 018] [REQ] [BridgeApplicationCommand]
                      └─[BasicCCSet]
                          target value: 255
19:30:35.636 CNTRLR   [Node 018] treating BasicCC Set as a report
19:30:35.637 CNTRLR   [Node 018] [~] [Basic] currentValue: 0 => 255                     [Endpoint 0]

At time 19:30:18.384 the sensor status was returned to 0. There's no command received from the device. The previous event received was at 14:30:18.376, which is almost exactly 5 hours.

AlCalzone commented 3 years ago

I've noticed that I no longer have a "Cover status" entity any longer. I suppose that means zwave-js has never seen a notification with that type yet? I haven't removed the cover since including it, although the act of including the node uses the tamper button, so after including just putting the cover back on should have triggered an event, so I'm not sure why it's missing.

That could be. These are V2 devices that don't allow discovering the available events. It could be that you need to trigger the event, it could be that HA requires https://github.com/zwave-js/node-zwave-js/issues/1899 to display them correctly.

I'm looking through my logs, is the idle time actually 5 hours instead of minutes?

WTF, let me check.

AlCalzone commented 3 years ago

Whoopsie: grafik

macf0x commented 3 years ago

Not sure I understand this 100%. Are you using the tilt sensor (binary sensor) for the door opening, and the cover (tamper) sensor to detect mail being dropped in?

Correct that is what I'm doing. I don't really need to know the the state of the letterbox (mail in) door, just that it was opened. I use the binary sensor to check when the rear door has been opened when collecting the mail. It's combined with automation an input/helper to to present a dashboard icon showing there is mail to be collected.

Letterbox3

macf0x commented 3 years ago

Updated:

zwavejs2mqtt: 2.4.1 zwave-js: 6.6.3

When running zwavejs2mqtt I can see the notifications, though only 1 value auto-idles However entities do not update with the latest release.

entityreg.txt Node24debug.txt

macf0x commented 3 years ago

zwavejs2mqtt.log

Not seeing either of these values auto idle.

macf0x commented 3 years ago

zwavejs_357.log Node 24 is the one not idling the values

AlCalzone commented 3 years ago

https://github.com/zwave-js/node-zwave-js/pull/2483

macf0x commented 3 years ago

Thanks. Will give that a shot. and report the results.

macf0x commented 3 years ago

Hi AlCalzone.

Thanks for making that adjustment to the config.

I note that config fix only idles 1 of the two values. 113-0-Home Security-Sensor is idles 113-0-Home Security-unknown does idles

This would be less of a problem if these came through as events in HA, but neither of these do as per https://www.home-assistant.io/integrations/zwave_js#node-events-notification https://zwave-js.github.io/node-zwave-js/#/api/node?id=quotnotificationquot

2021-05-02 14:35:57.076 INFO ZWAVE: Node 24: value updated: 113-0-Home Security-unknown 254 => 254
2021-05-02 14:36:00.082 INFO ZWAVE: Node 24: value updated: 113-0-Home Security-unknown 254 => 254
2021-05-02 14:36:03.635 INFO ZWAVE: Node 24: value updated: 48-0-Any true => false
2021-05-02 14:36:03.683 INFO ZWAVE: Node 24: value updated: 113-0-Home Security-Sensor status 0 => 2
2021-05-02 14:36:05.015 INFO ZWAVE: Node 24: value updated: 48-0-Any false => true
2021-05-02 14:36:05.036 INFO ZWAVE: Node 24: value updated: 113-0-Home Security-Sensor status 2 => 2

2021-05-02 14:41:05.046 INFO ZWAVE: Node 24: value updated: 113-0-Home Security-Sensor status 2 => 0
AlCalzone commented 3 years ago

These are emitted as events.. Not sure how you are listening for them, because the docs you shared are from the official integration while the logs are from zwavejs2mqtt.

kpine commented 3 years ago

"value updated" events are not events in HA, they are by default handled as state updates for entities. In special cases (as of version 2021.5) they can be treated as events. HA does not create sensor entities for these types of values because the metadata type is "any". The other notification values are all "number" type.

Also, have you re-interviewed the node? I do not have an "unknown" value. I have "113-0-Home Security-Cover status" and "113-0-Home Security-Sensor status".

AlCalzone commented 3 years ago

@kpine these are standard V2+ notifications and should arrive in HA as a notification event.

The unknown thing is a device problem - it actually sends a notification with the "unknown" event 0xfe.

kpine commented 3 years ago

these are standard V2+ notifications and should arrive in HA as a notification event.

Hmm, Home Security - Cover status and Home Security - Sensor status are both notification state variables, why would they be event notifications? Both zwavejs2mqtt and the websocket show these as value updated instead of value notification.

WSMessage(type=<WSMsgType.TEXT: 1>, data='{"type":"event","event":{"source":"node","event":"value updated","nodeId":23,"args":{"commandClassName":"Notification","commandClass":113,"endpoint":0,"property":"Home Security","propertyKey":"Sensor status","newValue":2,"prevValue":2,"propertyName":"Home Security","propertyKeyName":"Sensor status"}}}', extra='')

They are also showing up as values in zwavejs2mqtt (and HA).

 {
      "id": "23-113-0-Home Security-Cover status",
      "nodeId": 23,
      "commandClass": 113,
      "commandClassName": "Notification",
      "endpoint": 0,
      "property": "Home Security",
      "propertyName": "Home Security",
      "propertyKey": "Cover status",
      "propertyKeyName": "Cover status",
      "type": "any",
      "readable": true,
      "writeable": true,
      "label": "Home Security (property)",
      "stateless": false,
      "list": false,
      "value": 0,
      "lastUpdate": 1619818502386,
      "newValue": 0
    },
    {
      "id": "23-113-0-Home Security-Sensor status",
      "nodeId": 23,
      "commandClass": 113,
      "commandClassName": "Notification",
      "endpoint": 0,
      "property": "Home Security",
      "propertyName": "Home Security",
      "propertyKey": "Sensor status",
      "propertyKeyName": "Sensor status",
      "type": "any",
      "readable": true,
      "writeable": true,
      "label": "Home Security (property)",
      "stateless": false,
      "list": false,
      "value": 0,
      "lastUpdate": 1619976779138,
      "newValue": 0
    },

The CC V2 versions of these values are of "any" type instead of "number" types with states. Is V2 different in that regards?

Here's a CC V5 value of the same Cover status (different device), which HA understands.

{
      "id": "3-113-0-Home Security-Cover status",
      "nodeId": 3,
      "commandClass": 113,
      "commandClassName": "Notification",
      "endpoint": 0,
      "property": "Home Security",
      "propertyName": "Home Security",
      "propertyKey": "Cover status",
      "propertyKeyName": "Cover status",
      "type": "number",
      "readable": true,
      "writeable": false,
      "label": "Cover status",
      "stateless": false,
      "ccSpecific": {
        "notificationType": 7
      },
      "min": 0,
      "max": 255,
      "list": true,
      "states": [
        {
          "text": "idle",
          "value": 0
        },
        {
          "text": "Tampering, product cover removed",
          "value": 3
        }
      ],
      "value": 3,
      "lastUpdate": 1619818501954,
      "newValue": 3
    },

The unknown thing is a device problem - it actually sends a notification with the "unknown" event 0xfe.

Strange, guess I just haven't seen or noticed this notification yet.

AlCalzone commented 3 years ago

You're absolutely right, I confused them with events. The any type seems like an oversight on my part.

macf0x commented 3 years ago

These are emitted as events.. Not sure how you are listening for them, because the docs you shared are from the official integration while the logs are from zwavejs2mqtt.

I'm looking at the zwavejs2mqtt logs and also using a the dev tools event listener. I was of the understanding the official integration operated the same as zwavejs2mqtt. Am I wrong in this assumption? TBH I'm not really fussed if its a notifcation/HA event or entity/value/state. I can trigger an automation in HA from either. As it stand these notifications don't go anywhere, as far as I can tell.

"value updated" events are not events in HA, they are by default handled as state updates for entities. In special cases (as of version 2021.5) they can be treated as events. HA does not create sensor entities for these types of values because the metadata type is "any". The other notification values are all "number" type.

Thanks good to know. Is there a change in 2021.5 that may get some visibility on these notifications?

23:45:14.009 SERIAL » [ACK]                                                                   (0x06)
23:45:33.924 SERIAL « 0x0110000400180a7105070000ff07fe00008c                              (18 bytes)
23:45:33.925 SERIAL » [ACK]                                                                   (0x06)
23:45:33.926 DRIVER « [Node 024] [REQ] [ApplicationCommand]
                      └─[NotificationCCReport]
                          notification type:   Home Security
                          notification status: 255
                          notification event:  Unknown (0xfe)
23:45:33.928 CNTRLR   [Node 024] [+] [Notification] Home Security[unknown]: 254         [Endpoint 0]

Also, have you re-interviewed the node? I do not have an "unknown" value. I have "113-0-Home Security-Cover status" and "113-0-Home Security-Sensor status".

Reinterviewed

2021-05-04 00:53:10.350 INFO ZWAVE: Node 24: interview started
2021-05-04 00:53:10.419 INFO ZWAVE: Success zwave api call refreshInfo { success: true, message: 'Success zwave api call', result: undefined }
2021-05-04 00:53:10.947 INFO ZWAVE: Node 24: interview stage PROTOCOLINFO completed
2021-05-04 00:53:23.245 INFO ZWAVE: Node 24: value added: 113-0-Home Security-Cover status => 3
2021-05-04 00:53:30.825 INFO ZWAVE: Node 24: value updated: 113-0-Home Security-Cover status 3 => 3
2021-05-04 00:53:32.880 INFO ZWAVE: Node 24 is now awake
2021-05-04 00:53:33.083 INFO ZWAVE: Node 24: interview stage NODEINFO completed
2021-05-04 00:53:33.267 INFO ZWAVE: Node 24: value added: 132-0-wakeUpInterval => 3600
2021-05-04 00:53:33.268 INFO ZWAVE: Node 24: value added: 132-0-controllerNodeId => 1
2021-05-04 00:53:33.455 INFO ZWAVE: Node 24: value added: 114-0-manufacturerId => 265
2021-05-04 00:53:33.456 INFO ZWAVE: Node 24: value added: 114-0-productType => 8202
2021-05-04 00:53:33.456 INFO ZWAVE: Node 24: value added: 114-0-productId => 2562
2021-05-04 00:53:33.663 INFO ZWAVE: Node 24: value added: 134-0-libraryType => 6
2021-05-04 00:53:33.664 INFO ZWAVE: Node 24: value added: 134-0-protocolVersion => 3.52
2021-05-04 00:53:33.669 INFO ZWAVE: Node 24: value added: 134-0-firmwareVersions => 4.84
2021-05-04 00:53:35.400 INFO ZWAVE: Node 24: metadata updated: 132-0-wakeUpInterval
2021-05-04 00:53:35.589 INFO ZWAVE: Node 24: value updated: 132-0-wakeUpInterval 3600 => 3600
2021-05-04 00:53:35.590 INFO ZWAVE: Node 24: value updated: 132-0-controllerNodeId 1 => 1
2021-05-04 00:53:36.188 INFO ZWAVE: Node 24: value added: 128-0-level => 66
2021-05-04 00:53:36.189 INFO ZWAVE: Node 24: value added: 128-0-isLow => false
2021-05-04 00:53:36.374 INFO ZWAVE: Node 24: value added: 32-0-currentValue => 99
2021-05-04 00:53:36.555 INFO ZWAVE: Node 24: metadata updated: 48-0-Any
2021-05-04 00:53:36.561 INFO ZWAVE: Node 24: value added: 48-0-Any => true
2021-05-04 00:53:36.966 INFO ZWAVE: Node 24: value added: 113-0-Home Security-Sensor status => 2
2021-05-04 00:53:36.987 INFO ZWAVE: Node 24: interview stage COMMANDCLASSES completed
2021-05-04 00:53:36.991 INFO ZWAVE: Node 24: interview stage OVERWRITECONFIG completed
2021-05-04 00:53:37.038 INFO ZWAVE: Node 24: interview stage NEIGHBORS completed
2021-05-04 00:53:37.039 INFO ZWAVE: Node 24: interview stage COMPLETE completed
2021-05-04 00:53:37.047 INFO ZWAVE: Node 24: value added 24-32-0-currentValue => 99
2021-05-04 00:53:37.048 INFO ZWAVE: Node 24: value added 24-32-0-targetValue => undefined
2021-05-04 00:53:37.048 INFO ZWAVE: Node 24: value added 24-48-0-Any => true
2021-05-04 00:53:37.049 INFO ZWAVE: Node 24: value added 24-132-0-wakeUpInterval => 3600
2021-05-04 00:53:37.049 INFO ZWAVE: Node 24: value added 24-132-0-controllerNodeId => 1
2021-05-04 00:53:37.050 INFO ZWAVE: Node 24: value added 24-113-0-Home Security-Cover status => 3
2021-05-04 00:53:37.050 INFO ZWAVE: Node 24: value added 24-113-0-Home Security-Sensor status => 2
2021-05-04 00:53:37.051 INFO ZWAVE: Node 24: value added 24-128-0-level => 66
2021-05-04 00:53:37.051 INFO ZWAVE: Node 24: value added 24-128-0-isLow => false
2021-05-04 00:53:37.051 INFO ZWAVE: Node 24: value added 24-114-0-manufacturerId => 265
2021-05-04 00:53:37.052 INFO ZWAVE: Node 24: value added 24-114-0-productType => 8202
2021-05-04 00:53:37.052 INFO ZWAVE: Node 24: value added 24-114-0-productId => 2562
2021-05-04 00:53:37.052 INFO ZWAVE: Node 24: value added 24-134-0-libraryType => 6
2021-05-04 00:53:37.053 INFO ZWAVE: Node 24: value added 24-134-0-protocolVersion => 3.52
2021-05-04 00:53:37.053 INFO ZWAVE: Node 24: value added 24-134-0-firmwareVersions => 4.84
2021-05-04 00:53:37.061 INFO ZWAVE: Node 24: interview COMPLETED, all values are updated
2021-05-04 00:53:37.062 INFO ZWAVE: Node 24 ready: Vision Security - ZG8101 (Garage Door Tilt Sensor)
2021-05-04 00:53:38.146 INFO ZWAVE: Node 24 is now asleep

2021-05-04 00:55:11.985 INFO ZWAVE: Node 24: value added: 113-0-Home Security-unknown => 254
2021-05-04 00:55:11.987 INFO ZWAVE: Node 24: value added 24-113-0-Home Security-unknown => 254
2021-05-04 00:55:13.350 INFO ZWAVE: Node 24: value updated: 113-0-Home Security-unknown 254 => 254
2021-05-04 00:55:16.057 INFO ZWAVE: Node 24: value updated: 48-0-Any true => false
2021-05-04 00:55:16.138 INFO ZWAVE: Node 24: value updated: 113-0-Home Security-Sensor status 2 => 2
2021-05-04 00:55:18.500 INFO ZWAVE: Node 24: value updated: 48-0-Any false => true
2021-05-04 00:55:18.574 INFO ZWAVE: Node 24: value updated: 113-0-Home Security-Sensor status 2 => 2

This device also has a secondary contact that I have added a reed switch. This generates the 113-0-Home Security-unknown notification. I guess if you don't have the contact wired, you'll never see this notification as it doesn't appear in the interview.

{
      "id": "24-113-0-Home Security-unknown",
      "nodeId": 24,
      "commandClass": 113,
      "commandClassName": "Notification",
      "endpoint": 0,
      "property": "Home Security",
      "propertyName": "Home Security",
      "propertyKey": "unknown",
      "propertyKeyName": "unknown",
      "type": "any",
      "readable": true,
      "writeable": true,
      "label": "Home Security (property)",
      "stateless": false,
      "list": false,
      "value": 254,
      "lastUpdate": 1620055511987,
      "newValue": 254
    }

Node24 debug.txt

ZetaPhoenix commented 3 years ago

So I'm having similar issues with a ZD2105 door/window sensor showing the status for the device never changes after moving from the OZW 1.4. In my case binary_sensor.front_door_access_control_window_door_is_open never changes but I see the [19-113-0-Access Control-Door state] Door state change from 23 to 22 as expected when the door is opened.

kpine commented 3 years ago

@ZetaPhoenix Try restarting HA and see if the problem persists. If so, please open a new issue for this device and attach a network dump. I have two ZD2105 and they work fine.

ZetaPhoenix commented 3 years ago

Finally got around around to doing a master reset on the Zwave controller to clear a phantom node and after doing so I seems to be working fine. It was either loading it from cache or removing the phantom node that never got added that fixed it.

macf0x commented 3 years ago

As far as I can see these NotificationCCReports do not appears as events in Hass. Using developer tools and listening too zwave_js_notification or zwave_js_value_notification does not report any events.

In special cases (as of version 2021.5) they can be treated as events. HA does not create sensor entities for these types of values because the metadata type is "any". The other notification values are all "number" type.

This is a deviation from the behaviour with OZW. Not that OZW is the standard/bar that must be met, but such issues are migration blockers. Really HA should be passing events through for any thing that isn't handled as a state/Value update.

There are always going to be devices that are nonconformant or not documented very well. There needs to be a bit more flexibility is handling such devices.

kpine commented 3 years ago

As far as I can see these NotificationCCReports do not appears as events in Hass. Using developer tools and listening too zwave_js_notification or zwave_js_value_notification does not report any events.

This is expected. These Z-Wave Notification types are state variables, not events. Thus they are reflected as value updates not notifications. In HA, value updates are associated with entity state changes, where as notifications are events.

In special cases (as of version 2021.5) they can be treated as events. HA does not create sensor entities for these types of values because the metadata type is "any". The other notification values are all "number" type.

This is a deviation from the behaviour with OZW. Not that OZW is the standard/bar that must be met, but such issues are migration blockers. Really HA should be passing events through for any thing that isn't handled as a state/Value update.

The notification sensor behavior is practically the same for the zwave_js integration as it is with ozw integration, both of which are improvements over the zwave integration. What is being discussed here is a bug with the metadata type for these values that effects the sensor entity discovery. The incorrect metadata comes from either node-zwave-js or the zwave-js-server. Addressing the root cause will allow HA to discover the entity properly.

macf0x commented 3 years ago

Thanks kpine for the reply, Originally that was the intent of the issue, but through troubleshooting it was revealed this was a notificationCCreport and should be treated as an event, which it appears to be in ZWJS, however HA is dropping/ignoring it.

As this is a notification, it doesn't make sense for it to be entity/value. zwave-js is reporting this as an event, but I assume it's being ignored.

23:45:33.926 DRIVER « [Node 024] [REQ] [ApplicationCommand]
                      └─[NotificationCCReport]
                          notification type:   Home Security
                          notification status: 255
                          notification event:  Unknown (0xfe)

The notification sensor behavior is practically the same for the zwave_js integration as it is with ozw integration, both of which are improvements over the zwave integration.

In the case of this issue, there is a difference as previously this Notification resulted in an entity. ZWJS team believe that should be the case now and handling it as an event.

I'm happy to take this up with ZWJS, but we seem to be bouncing back and forth, on how this should be handled.

kpine commented 3 years ago

I think there is a misunderstanding. None of these are events at the node-zwave-js to HA interface. These are all values with incorrect metadata. Review your own logs: https://github.com/home-assistant/core/issues/46982#issuecomment-831341082