home-assistant / core

:house_with_garden: Open source home automation that puts local control and privacy first.
https://www.home-assistant.io
Apache License 2.0
69.79k stars 28.93k forks source link

Z-Wave JS Integration Configuration Bug - Official Z-Wave JS Add-On detects controller but Integration does not #119113

Open btbutts opened 4 weeks ago

btbutts commented 4 weeks ago

The problem

I recently upgraded to a Z-Wave 800 series controller. I'm using the Zooz ZAC93 on a Raspberry Pi 5. I've been able to configure HA to use the Z-Wave GPIO HAT and when I configure the Official Z-Wave JS add-on via the add-on store, it sees /dev/ttyAMA0 without an issue.

However, when viewing the Z-Wave JS Integration via the Devices & services page, configuring it (Add new Hub) and leave the default option "Use the Z-Wave JS Supervisor add-on" checked, the Z-Wave JS Integration only shows /dev/ttyAMA10 and two Sonoff ZBDongle-E sticks I'm using for Zigbee and Thread. The /dev/ttyAMA0 is missing. This is of course a problem as for testing since I actually tried to setup a brand new HA instance on a new Home Assistant install (I'm running HASSOS), where I would then need to allow the Z-Wave JS Integration setup wizard to generate new Keys for the Legacy S0 and newer S2 and S2 LR networks.

Without the Z-Wave JS integration showing /dev/ttyAMA0, even though the official Z-Wave JS Add-On does, the Z-Wave JS Integration is unable to generate the keys I previously mentioned since it doesn't have a valid controller selected. I can force my way through this step by temporarily selecting /dev/ttyAMA10, which will of course fail (that's just the RPi5's console output) but it does allow the Integration to generate the required keys in the Official (Supervisor) Z-Wave JS Add-on. Then I need only to modify Z-Wave JS Add-on afterwards to use /dev/ttyAMA0 once again and restart the Add-On. After that. The respective Z-Wave integration is updated, and I can now see my controller.

Furthermore, I've been able to confirm that HA can successfully see my controller because I can see in the Z-Wave JS add-on logs that it is successfully connecting to the controller, scanning the nodes, and the controller information is visible in the integration config screen. I even used Simplicity Studio to import my Z-Wave network from a 500 series controller into the new 800 series controller (ZAC93), changed the keys appropriately in HA, and then watched as all of my nodes became visible in Home Assistant after a restart. I could control them too. The problem is that when I would try to add new nodes via the Z-Wave integration, that would fail. There would be two simple logs each time, regardless of the type of node I tried to add:

Z-WAVE: Controller status: Secure inclusion started
Z-WAVE: Controller status: Inclusion failed
Z-WAVE: Controller status: Inclusion stopped

Now if I attempt to turn off a light, dim a light, etc... of a node that's already in the Z-Wave network. I can do that just fine. I even tried a firmware upgrade of the controller just to verify that the RPi5 has a valid I2C connection with the controller. the UART and I2C are working. The add-on seems to be working. It's the integration that's configuration doesn't seem to recognize the controller installed at /dev/ttyAMA0 and thus I believe where the problems start.

Just for fun, I decided to remove the official Z-Wave JS add-on and integration and try out the Unofficial Z-Wave JS UI add-on just to see what would happen. It, just as the official add-on, has no issue connecting to the controller installed at /dev/ttyAMA0. I was able to get the Official Z-Wave integration to connect to the "Z-Wave JS UI" add-on as well. However, when attempting to add new nodes via the integration, I saw the same behavior as I had with the official Z-Wave JS Supervisor add-on. No change at all. When I tried it with an existing Z-Wave network, I could control my nodes as well. It would appear that both add-ons seem to do their job just fine. However, the integration is where the issue exists.

There seems to be some issue with the Z-Wave integration using controllers at /dev/ttyAMA0 for some reason, even though the add-on works with them just fine. There's no controller connection issues in the logs whatsoever actually.

What version of Home Assistant Core has the issue?

core-2024.6.1

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

Z-Wave JS

Link to integration documentation on our website

https://www.home-assistant.io/integrations/zwave_js/

Diagnostics information

{ "home_assistant": { "installation_type": "Home Assistant OS", "version": "2024.6.1", "dev": false, "hassio": true, "virtualenv": false, "python_version": "3.12.2", "docker": true, "arch": "aarch64", "timezone": "America/Chicago", "os_name": "Linux", "os_version": "6.6.28-haos-raspi", "supervisor": "2024.06.0", "host_os": "Home Assistant OS 12.3", "docker_version": "25.0.5", "chassis": "embedded", "run_as_root": true }, "custom_components": { "hacs": { "documentation": "https://hacs.xyz/docs/configuration/start", "version": "1.34.0", "requirements": [ "aiogithubapi>=22.10.1" ] }, "ttlock": { "documentation": "https://github.com/jbergler/hass-ttlock", "version": "v0.6.2", "requirements": [ "pydantic" ] }, "tuya_ble": { "documentation": "https://www.home-assistant.io/integrations/tuya_ble", "version": "0.1.8", "requirements": [ "tuya-iot-py-sdk==0.6.6", "pycountry==22.3.5" ] }, "huesyncbox": { "documentation": "https://github.com/mvdwetering/huesyncbox", "version": "2.1.0", "requirements": [ "aiohuesyncbox==0.0.27" ] }, "ble_monitor": { "documentation": "https://github.com/custom-components/ble_monitor", "version": "12.11.2", "requirements": [ "pycryptodomex>=3.19.1", "janus>=1.0.0", "aioblescan>=0.2.14", "btsocket>=0.2.0", "pyric>=0.1.6.3" ] }, "smartthinq_sensors": { "documentation": "https://github.com/ollo69/ha-smartthinq-sensors", "version": "0.39.1", "requirements": [ "pycountry>=23.12.11", "xmltodict>=0.13.0", "charset_normalizer>=3.2.0" ] } }, "integration_manifest": { "domain": "zwave_js", "name": "Z-Wave", "codeowners": [ "home-assistant/z-wave" ], "config_flow": true, "dependencies": [ "http", "repairs", "usb", "websocket_api" ], "documentation": "https://www.home-assistant.io/integrations/zwave_js", "integration_type": "hub", "iot_class": "local_push", "loggers": [ "zwave_js_server" ], "quality_scale": "platinum", "requirements": [ "pyserial==3.5", "zwave-js-server-python==0.56.0" ], "usb": [ { "vid": "0658", "pid": "0200", "known_devices": [ "Aeotec Z-Stick Gen5+", "Z-WaveMe UZB" ] }, { "vid": "10C4", "pid": "8A2A", "description": "z-wave", "known_devices": [ "Nortek HUSBZB-1" ] } ], "zeroconf": [ "_zwave-js-server._tcp.local." ], "is_built_in": true }, "setup_times": {}, "data": { "versionInfo": { "driverVersion": "12.5.5", "serverVersion": "1.35.0", "minSchemaVersion": 0, "maxSchemaVersion": 35 }, "entities": [], "state": { "nodeId": 1, "index": 0, "status": 4, "ready": true, "isListening": true, "isRouting": true, "manufacturerId": 634, "productId": 1553, "productType": 4, "firmwareVersion": "1.10", "deviceConfig": { "filename": "/data/db/devices/0x027a/zac93.json", "isEmbedded": true, "manufacturer": "Zooz", "manufacturerId": 634, "label": "ZAC93", "description": "800 Series Long Range GPIO Module", "devices": [ { "productType": 4, "productId": 1553 } ], "firmwareVersion": { "min": "0.0", "max": "255.255" }, "preferred": false, "metadata": { "comments": [] } }, "label": "ZAC93", "interviewAttempts": 1, "isFrequentListening": false, "maxDataRate": 100000, "supportedDataRates": [ 40000, 100000 ], "protocolVersion": 3, "supportsBeaming": true, "supportsSecurity": false, "nodeType": 0, "deviceClass": { "basic": { "key": 2, "label": "Static Controller" }, "generic": { "key": 1, "label": "Remote Controller" }, "specific": { "key": 0, "label": "Unused" }, "mandatorySupportedCCs": [], "mandatoryControlledCCs": [ 32 ] }, "interviewStage": "Complete", "deviceDatabaseUrl": "https://devices.zwave-js.io/?jumpTo=0x027a:0x0004:0x0611:1.10", "statistics": { "commandsTX": 0, "commandsRX": 0, "commandsDroppedRX": 0, "commandsDroppedTX": 0, "timeoutResponse": 0 }, "isControllerNode": true, "keepAwake": false, "protocol": 0, "values": { "1-32-0-currentValue": { "endpoint": 0, "commandClass": 32, "commandClassName": "Basic", "property": "currentValue", "propertyName": "currentValue", "ccVersion": 0, "metadata": { "type": "number", "readable": true, "writeable": false, "label": "Current value", "min": 0, "max": 99, "stateful": true, "secret": false } } }, "endpoints": { "0": { "nodeId": 1, "index": 0, "deviceClass": { "basic": { "key": 2, "label": "Static Controller" }, "generic": { "key": 1, "label": "Remote Controller" }, "specific": { "key": 0, "label": "Unused" }, "mandatorySupportedCCs": [], "mandatoryControlledCCs": [ 32 ] }, "commandClasses": [] } } } } }

Example YAML snippet

device: /dev/ttyAMA0
log_level: info
log_to_file: false
log_max_files: 8
soft_reset: Automatic

Anything in the logs that might be useful for us?

Starting server on <all interfaces>:3000
2024-06-08T03:26:53.729Z CNTRLR   [Node 001] Beginning interview - last completed stage: None
2024-06-08T03:26:53.729Z CNTRLR   [Node 001] new node, doing a full interview...
2024-06-08T03:26:53.730Z CNTRLR » [Node 001] querying protocol info...
ZwaveJS server listening on <all interfaces>:3000
2024-06-08T03:26:53.835Z CNTRLR « [Node 001] received response for protocol info:
                                  basic device class:    Static Controller
                                  generic device class:  Remote Controller
                                  specific device class: Unused
                                  node type:             Controller
                                  is always listening:   true
                                  is frequent listening: false
                                  can route messages:    true
                                  supports security:     false
                                  supports beaming:      true
                                  maximum data rate:     100000 kbps
                                  protocol version:      3
2024-06-08T03:26:53.835Z CNTRLR   [Node 001] Interview stage completed: ProtocolInfo
2024-06-08T03:26:53.852Z CNTRLR   [Node 001] Embedded device config loaded
2024-06-08T03:26:53.853Z CNTRLR   [Node 001] Interview stage completed: OverwriteConfig
2024-06-08T03:26:53.853Z CNTRLR   [Node 001] Interview completed
2024-06-08T03:26:53.854Z CNTRLR   [Node 001] The node is ready to be used
2024-06-08T03:26:53.854Z CNTRLR   All nodes are ready to be used
2024-06-08T03:26:53.855Z CNTRLR   [Node 001] The node is alive.
2024-06-08T03:26:53.856Z CNTRLR   Interviewing nodes and/or determining their status: 
New client
Client disconnected
Code 1000: 
New client
2024-06-08T03:30:40.389Z CNTRLR   Starting inclusion process with strategy Default...
2024-06-08T03:30:40.401Z CNTRLR   The controller is now ready to add nodes
2024-06-08T03:31:02.138Z CNTRLR   handling add node request (status = NodeFound)
2024-06-08T03:31:24.579Z CNTRLR   handling add node request (status = Failed)
2024-06-08T03:31:24.580Z CNTRLR   Adding the node failed
2024-06-08T03:31:24.580Z CNTRLR   stopping inclusion process...
2024-06-08T03:31:24.588Z CNTRLR   stopping inclusion process...
2024-06-08T03:31:24.591Z CNTRLR   The inclusion process was stopped
2024-06-08T03:31:24.598Z CNTRLR   The inclusion process was stopped

Additional information

No response

home-assistant[bot] commented 4 weeks ago

Hey there @home-assistant/z-wave, mind taking a look at this issue as it has been labeled with an integration (zwave_js) you are listed as a code owner for? Thanks!

Code owner commands Code owners of `zwave_js` can trigger bot actions by commenting: - `@home-assistant close` Closes the issue. - `@home-assistant rename Awesome new title` Renames the issue. - `@home-assistant reopen` Reopen the issue. - `@home-assistant unassign zwave_js` Removes the current integration label and assignees on the issue, add the integration domain after the command. - `@home-assistant add-label needs-more-information` Add a label (needs-more-information, problem in dependency, problem in custom component) to the issue. - `@home-assistant remove-label needs-more-information` Remove a label (needs-more-information, problem in dependency, problem in custom component) on the issue.

(message by CodeOwnersMention)


zwave_js documentation zwave_js source (message by IssueLinks)

btbutts commented 4 weeks ago

One thing I'll add is that I know that when using a USB controller, its recommended to specify the controller path via the systemd link generated at /dev/serial/by-path/ via udev rules (IE: the rules located in /usr/lib/udev/rules.d/60-serial.rules). The challenge with this with GPIO HAT Z-Wave controllers is that even though the RPi5 UARTs are connected to a PCI device in the CPU, devices at /dev/ttyACM0 are not included in the rules, thus no path is generated. Specifically, the current lines for this are:

SUBSYSTEMS=="usb", IMPORT{builtin}="usb_id", IMPORT{builtin}="hwdb --subsystem=usb"
SUBSYSTEMS=="pci", ENV{ID_BUS}=="", ENV{ID_BUS}="pci", \
  ENV{ID_VENDOR_ID}="$attr{vendor}", ENV{ID_MODEL_ID}="$attr{device}", \
  IMPORT{builtin}="hwdb --subsystem=pci"

# /dev/serial/by-path/, /dev/serial/by-id/ for USB devices
KERNEL!="ttyUSB[0-9]*|ttyACM[0-9]*", GOTO="serial_end"

With the first line, USB UART devices will have a path generated. The second line allows for PCI devices. However, since the KERNEL line omits ttyAMA[0-9]*, I'm thinking that this is why no system path is generated by the udev rules for GPIO UART devices. I've thought about potentially modifying this file for testing but the filesystem is readonly this is something I'd need to validate with an isolated Home Assistant lab running HASSOS.

My thought is that perhaps one could:

  1. Modify the last line (as copied above) to include ttyAMA[0-9]*
  2. Verify that a system path is generated by the udev rules at /dev/serial/by-path/
  3. If number 2 above is successful, identify if the Z-Wave JS Add-On and Integration alike detect the new path and see if that improves the behavior I'm seeing.