Network Died and Can't Recreate

wraith1385 commented 2 years ago

Is your problem within Home Assistant (Core or Z-Wave JS Integration)?

NO, my problem is NOT within Home Assistant or the ZWave JS integration

Is your problem within ZWaveJS2MQTT?

NO, my problem is NOT within ZWaveJS2MQTT

Checklist

[X] I have checked the troubleshooting section and my problem is not described there.
[X] I have read the changelog and my problem was not mentioned there.

Describe the bug

Woke up a few days ago and none of my z-wave devices were responding. Thought it was a bad stick so I ordered a replacement and am getting the same results.

My setup is a rasberry pi 4 with zwavejs2mqtt running in a docker container and then HomeAssistant points to that. At this point I have tried 3 different zooz zst10 700 sticks as well as an aeotec z-stick 7 plus. I have upgraded the firmware on all of the sticks to 7.17.2. I have factory reset the sticks, blown away the docker container and recreated it, switched to try and use the zwavejs addon in HA, and switched to try the zwavejs2mqtt addon in HA.

During all of this I thought I had an nvm backup of the original stick, but apparently not. However, I have factory reset a few different zwave devices in my house (a switch, a sensor, a plug, and the garage door controller) to try to begin recreating the network.

No matter which configuration I try, the sticks don't see anything in both inclusion and exclusion mode. I've turned logging up to silly and all I see are the API calls to start and end the inclusion or exclusion modes.

Device information

Have tried both the Aeotec z-stick 7 plus as well as a few zooz zst10 700's. Firmware on all of them was upgraded to 7.17.2.

How are you using `node-zwave-js`?

[X] zwavejs2mqtt Docker image (latest)
[ ] zwavejs2mqtt Docker image (dev)
[ ] zwavejs2mqtt Docker manually built (please specify branches)
[ ] ioBroker.zwave2 adapter (please specify version)
[X] HomeAssistant zwave_js integration (please specify version)
[ ] pkg
[ ] node-red-contrib-zwave-js (please specify version, double click node to find out)
[ ] Manually built from GitHub (please specify branch)
[ ] Other (please describe)

Which branches or versions?

version: node-zwave-js branch: 10.0.4 zwavejs2mqtt branch: latest (7.1.0) HA zwave_js integration: 0.1.70

Don't run both zwavejs2mqtt and the HA zwavejs integration at the same time, but during the troubleshooting process I was trying everything.

Did you change anything?

no

If yes, what did you change?

No response

Did this work before?

Yes (please describe)

If yes, where did it work?

House had a successful zwave mesh with over 20 mixed devices (locks, switches, outlets, etc) zwavejs2mqtt-store 2.zip

Attach Driver Logfile

zwavejs_2022-09-09.log zwavejs_2022-09-08.log

AlCalzone commented 2 years ago

Where are you located? USA? EU? Is your stick on an extension cord? https://zwave-js.github.io/node-zwave-js/#/troubleshooting/connectivity-issues?id=general-troubleshooting

Also, did you do an NVM backup before factory-resetting?

wraith1385 commented 2 years ago

Located in USA Stick is on an extension cord. See photo

I thought I did an NVM backup, but it must not have ever completed so that's all gone, unless it can somehow be recreated from the json files in the store folder.

AlCalzone commented 2 years ago

No unfortunately it can't be recreated. You could try moving the stick a bit further away from the other radios, maybe bring one device closer for testing. Other than that I don't see anything wrong.

wraith1385 commented 2 years ago

I held the stick out farther away and brought one of the sensors right next to it and got the same results. Also some of the switches and outlets I was testing with are only 5 or so feet away. It's honestly like my house has turned into some kind of z-wave dead zone.

kpine commented 2 years ago

What about the controller's region and power levels? Are they set correctly?

AlCalzone commented 2 years ago

According to the log, +9.9dBm TX power, region USA. So there should be at least some signal.

wraith1385 commented 2 years ago

Yeah, this is what the GUI shows

kpine commented 2 years ago

I don't know if it matters, or what the impact is, but those power levels are nowhere close to the recommended ones for a 700-series stick. What affect does that have?

AlCalzone commented 2 years ago

In theory, the powerlevel setting here should make the stick "shout louder", and the 2nd value is for calibrating what the actual power is. Different sticks seem to have different defaults here.

@wraith1385 You could try going to the combination of values that @kpine's screenshot shows. Maybe that changes things, maybe not.

Elsewhere I've tested the settings for my EU Z-Stick 7 and they had no noticeable effect. But that might be a different issue.

wraith1385 commented 2 years ago

Changed the values to those in the screenshot. Same results.

AlCalzone commented 2 years ago

At this point I'm afraid you need a Zniffer (UZB3 flashed with special firmware) to inspect the RF traffic and figure out if it's the controller not receiving or the devices not sending.

wraith1385 commented 2 years ago

Ok, I've just ordered a UZB3. Can someone on here help me with what I'm looking for when I get it and flash it? I've never delved this deep into z-wave before.

AlCalzone commented 2 years ago

Here's a description how you can convert it: https://community.silabs.com/s/article/z-wave-500-converting-a-uzb3-controller-to-a-zniffer?language=en_US

The necessary tools are linked there somewhere, but if you can't find them, I should also have them somewhere.

AlCalzone commented 2 years ago

Apparently, the Z-Wave Programmer is part of the Z/IP Gateway SDK, which can be downloaded through Simplicity Studio:

zw_programmer

A programming tool with source distributed in Z/IP Gateway SDK. It communicates with Z-Wave controller via UART interface, and can be used for Z-Wave controller OTW update. Please install the latest Z/IP Gateway SDK 7.18 from Simplicity Studio v5.

After installing Z/IP Gateway package on Linux, the zw_programmer is typically located at /usr/local/bin

The source is available in Z/IP Gateway SDK package: C:\SiliconLabs\SimplicityStudio\v5\developer\sdks\zwave\zip_gateway\v7.18.0\Source\zipgateway-7.18.00-Source.tar\zipgateway-7.18.00-Source\usr\local\systools\zw_programmer

kpine commented 2 years ago

zw_programmer is a command line tool for doing OTW updates of the 700-series controllers and NVM backups and restores. It won't work for flashing the Zniffer firmware on a UZB3. It's also painful to compile and install (I wish they'd just make it an available download for everyone, by far the easiest way to update a 700 stick!). You need the "Z-Wave Programmer" (or also called PC Programmer) windows application to do that.

I find this guide to be the easiest to follow: https://bishopfox.com/blog/set-up-zniffer-for-z-wave

Silicon Labs is constantly re-designing their website, but the link there for the downloads is still valid. The downloads can be easy to miss, don't go to the "Software & Tools" tab, stay on the Overview tab. Scroll down to the right and look for the "Z-Wave Protocol SDK Downloads" section. You'll need to create a Silicon Labs account to download them. You need both Z-Wave Programmer and Zniffer downloads. You won't need Simplicity Studio at all if you use these.

The Zniffer download from above includes the zniffer firmware. The Zniffer downloaded with Simplicity Studio does not include the UZB3 firmware. Simplicity Studio does have a newer version of the Zniffer application though.

For anyone else that stumbles upon this comment, I did find a community post with a sniffer_ZW050x_USBVCP.hex file download. In case the Zniffer download linked above gets lost in another Silicon Labs website redesign, you may still be able to get the hex file here. I did confirm it's the same exact file in the old Zniffer download.

wraith1385 commented 2 years ago

Ok, I got my zniffer flashed and ready. Can someone direct me as to what to test and what I'm looking for?

AlCalzone commented 2 years ago

Start by zniffing the stuff that doesn't work. I can take a look tomorrow.

wraith1385 commented 2 years ago

I am so unbelievably confused. All of a sudden items are working. I've attached the trace file. Can you take a look when you get a chance and tell me if anything looks wrong?

trace.zip

wraith1385 commented 2 years ago

My network has now been stable for over a week. No idea what changed.

zwave-js-assistant[bot] commented 2 years ago

This issue has not seen any recent activity and was marked as "stale 💤". Closing for housekeeping purposes... 🧹

Feel free to reopen if the issue persists.

zwave-js-assistant[bot] commented 2 years ago

This issue has not seen any recent activity and was marked as "stale 💤". Closing for housekeeping purposes... 🧹

Feel free to reopen if the issue persists.

zwave-js / node-zwave-js