zwave-js / node-zwave-js

Z-Wave driver written entirely in JavaScript/TypeScript
https://zwave-js.github.io/node-zwave-js/
MIT License
749 stars 598 forks source link

800 Series Z-Wave Controller Support Tracker #5257

Closed blhoward2 closed 6 months ago

blhoward2 commented 1 year ago

Update (2024-04-02)

800 series controllers are fully supported as of https://github.com/zwave-js/node-zwave-js/releases/tag/v12.4.4.

NVM Backup/Restore requires a firmware based on Z-Wave SDK 7.19.0 or higher.

Experimental Long Range support is available in https://github.com/zwave-js/node-zwave-js/releases/tag/v12.5.0, which is being released now

marcus-j-davies commented 10 months ago

It's not that it's not a priority, but it's not going to be as high of a priority as alcalzone's kids. He's on parental leave. A contributor already did a ton of work towards LR, but he presumably has a life too so it's taken some time to finalize, as can be expected.

What @blhoward2 didn't tell you, is that @AlCalzone is training his little ones to manage the project when he is busy with other things.

Who do you think zwave-js-bot is 😉

darkbasic commented 10 months ago

Who do you think zwave-js-bot is

Too much incognito, my bot has a 100% chance of people mistaking it for me: https://github.com/darkbasic/mikro-orm-dataloaders/pull/14 :)

jum0n commented 10 months ago

_"We have verified with Silabs, and they have promised a firmware update that will provide NVM backup and restore functionality between chip versions with an ETA of approximately 4 weeks. If you'd like, I can add you to the waitlist and we can reach out once we have confirmed the firmware fully resolves the migration issue?"

I would guess this is only if you have a previous gen Zooz to a recent Zooz? I have a 500 series Aeotec running and a 800 series Zooz waiting to take its place. Also not sure if there is any hurry since I think Zwave JS UI doesnt support LR yet.

ExodusC commented 10 months ago

I would guess this is only if you have a previous gen Zooz to a recent Zooz? I have a 500 series Aeotec running and a 800 series Zooz waiting to take its place. Also not sure if there is any hurry since I think Zwave JS UI doesnt support LR yet.

Someone can correct me if I am wrong here, but I was pretty sure all of these differently branded Z-Wave controllers are literally the same hardware underneath, just in a different case.

My ZST10 in Home Assistant just shows up as a generic Silicon Labs Z-Wave 700 series controller. It was my understanding they all run the same firmware images. I could totally be wrong here though.

kpine commented 10 months ago

My ZST10 in Home Assistant just shows up as a generic Silicon Labs Z-Wave 700 series controller. It was my understanding they all run the same firmware images. I could totally be wrong here though.

This is not true. There are only a few Z-Wave SOCs, but the boards and packaging can be different. Zooz controllers use different chips than SiLabs UZB and Aeotec. At least the original Zooz 700 was a re-branded Hank. Aoetec and UZB are the same SOC but Aeotec has designed their own PCB and antennas, etc.

Your Zooz only shows as a generic Silicon Labs stick because they were using the same product identifiers. Recent Zooz products have customized the firmware to include their own IDs and other minor changes.

Z-wave.me goes even further and highly customizes the firmware.

Of course, the above products are all based on the same Silicon Labs SDK.

Hedda commented 10 months ago

My ZST10 in Home Assistant just shows up as a generic Silicon Labs Z-Wave 700 series controller. It was my understanding they all run the same firmware images. I could totally be wrong here though.

This is not true. There are only a few Z-Wave SOCs, but the boards and packaging can be different. Zooz controllers use different chips than SiLabs UZB and Aeotec. At least the original Zooz 700 was a re-branded Hank. Aoetec and UZB are the same SOC but Aeotec has designed their own PCB and antennas, etc.

FYI, USB Discovery identification in Home Assistant can only be done if the manufacturer has written a unique product description to the EEPROM of the USB-to-Serial bridge/converter chip on the board (which most manufacturers forget or miss), and does not depend on the Z-Wave chip used, see:

https://community.home-assistant.io/t/community-help-wanted-to-whitelist-all-compatible-zigbee-and-z-wave-dongles-adapters-for-automatic-usb-discovery-in-home-assistant/344412

So if you want automatic USB discovery then you first need to convince manufacturer to actually write a unique product description to the EEPROM of the USB-to-Serial bridge/converter chip on the board so that can be read and added to the USB discovering whitelisting for Z-Wave adapters in Home Assistant.

jtbraun commented 8 months ago

Thanks to @AlCalzone taking it over the finish line, there's driver support for LR controllers/nodes in https://github.com/zwave-js/node-zwave-js/releases/tag/v12.5.0-beta.0.

gabemcg commented 8 months ago

Thanks to @AlCalzone taking it over the finish line, there's driver support for LR controllers/nodes in https://github.com/zwave-js/node-zwave-js/releases/tag/v12.5.0-beta.0.

Excited to test this out! It looks like the "test" tag for zwave-js-ui docker container is only on 12.4.1 is there a way to run 12.5.0-beta0 within the UI container or do I just need to be patient and wait?

jtbraun commented 8 months ago

@gabemcg you'll need to wait. Some small changes (https://github.com/zwave-js/zwave-js-ui/issues/3536#issuecomment-1900550730) need to be made to actually include a long range node (IE: tell the driver to include the node as long range).

AlCalzone commented 8 months ago

@robertsLando will make a test release soon. I advise against using this productively for now though due to further changes that are necessary. At the very least we need to use two separate S2 keys for Long Range.

TheKorn2 commented 8 months ago

So should a nvm backup from a zst-39 to another zst-39 work or fail at this point? (...because mine are failing. The backup gives me a 40k file, but the restore doesn't work.)

AlCalzone commented 8 months ago

Confirmed Not Working: NVM Restore

officialh1 commented 8 months ago

Being my first move into Home Assistant and using zwave-js, I need to gauge my experience a bit. I have 3 of these 800 series controllers controlling in total 144 zwave devices. No more than 53 devices per controller. They are running in docker on raspberry pis ((1)3 and (2)4/8gb if that info matters at all). I have them on the lastest release and started with 12.4.0, so yes I am very new to this.

I get dead nodes frequently enough when they are not really dead. They may timeout if it just turned dead but waiting about 60 seconds or so, they ping back fine. This tends to happen in busy events (all off, for example). I have had to include ping events before executing door lock events with delays added in. This might likely be a controller firmware issue, not sure.

It would be nice to have an option to send a command even if zwave-js thinks it's dead. I can do that from zwave-js UI (without pinging, just turn on the light or unlock door) but Home Assistant will not. Maybe that should be filed under Home Assistant then. Sorry just trying to get my bearings here.

There might be something here for dead nodes too where it's a busy controller or too much signalling going on? I have various vendors and it's not just specific ones. I think zwave-js could handle dead nodes better when it's more likely just a timeout and trying again in 30,60,120,300 second intervals (with random variation) for just the dead nodes to try and not report dead nodes? Maybe a dead pending state vs on/off dead state?

blhoward2 commented 8 months ago

That sounds more like a network congestion issue. Run a network heal. And watch the zwavejs log at the debug level (or use a zniffer) and see what's going across the mesh. Often this is caused by too many devices reporting sensor values way too frequently instead of based on a threshold change. Some devices are configured by default to report way more than they should.

Splitting them by controller may actually be hurting things as well. They all clog the same frequency and you're going to get way more collisions without one controller being able to prioritize traffic to get ACKs out so devices stop repeating traffic.

AlCalzone commented 8 months ago

@officialh1 what you describe could be lots of things. Happy to take a look and give you some hints. That doesn't belong in this issue though. Please open a new discussion here and provide the requested info, including what you describe above.

joshs85 commented 8 months ago

I was able to do an NVM backup using their 800 series LR GPIO module for HomeAssistant Yellow

vignez2991 commented 8 months ago

When can I expect backup and restore functionality for the 800 Series? It shows that the NVM backup was successfully created, and the resulting file size is non-zero. Does this indicate that the backup process is working as intended?

TheKorn2 commented 8 months ago

I don't know that I would use the word expect with any of this; that sounds demanding for something you (and I) haven't paid for.

If you really need to back up/restore your NVM now now now, you can use simplicity studio. Yes it's sub-optimal, but you're not completely SOL either.

AlCalzone commented 8 months ago

Backup is working, the zero file size was a firmware bug.

Restore isn't yet, since Z-Wave JS tries to parse the NVM to know whether it needs to convert it or not. I won't make any promises when this will be supported, as I'm on parental leave until April (see pinned issues).

There is also a method to restore without this check (restoreNVMRaw), but I don't know if Z-Wave JS UI exposes that. @robertsLando ?

robertsLando commented 8 months ago

There is also a method to restore without this check (restoreNVMRaw), but I don't know if Z-Wave JS UI exposes that.

yeah it's the restore nvm action but you have to check the Use Raw flag

chrispatin commented 7 months ago

Based on the release notes it looks like the restore functionality is working. I backed up an Aeotec 700 controller, and replaced it with a Zooz 800 series and worked flawless. I had some distant nodes where the responsiveness was mediocre. This was greatly improved with the upgrade. (~100 devices)

AlCalzone commented 7 months ago

Correct. It's all there now, except Long Range support, which is only in an experimental branch for now.

zacs commented 7 months ago

If going from 500->800, will a plain NVM backup/restore work, or is the right thing to do still to use nvmedit to convert and restore?

AlCalzone commented 7 months ago

nvmedit is automatically done behind the scenes. As long as the 500 series is running SDK 6.61 or newer, backup+restore should just work.

Hedda commented 7 months ago

Does backup and restore to migrate going from 700 series to 800 series dongle also work if using the built-in Z-Wave JS add-on in Home Assistant (running Home Assistant Operating System or Home Assistant Supervised)?

https://www.home-assistant.io/integrations/zwave_js/

PS: Living in the European Union so the new Long Range standard is not certified here yet anyway as I understand.

kpine commented 7 months ago

Does backup and restore to migrate going from 700 series to 800 series dongle also work if using the built-in Z-Wave JS add-on in Home Assistant (running Home Assistant Operating System or Home Assistant Supervised)?

No, HA does not support NVM operations. You could use the raw websocket API directly if you know what you're doing.

Hedda commented 7 months ago

No, HA does not support NVM operations. You could use the raw websocket API directly if you know what you're doing.

Ah, are there any plans to extend the built-in Z-Wave JS add-on in Home Assistant to support CLI commands for NVM operations? …maybe similar to how their Zigbee integration does it via zigpy-cli? https://github.com/zigpy/zigpy-cli

skynet01 commented 7 months ago

Is there any specific guide someone can point me to do a 500->800 update? Would like to update my Aeotek 500 (ZW090) to Zooz 800 usb stick but afraid to do the plunge until someone confirms it working :)

mkosmo commented 7 months ago

Is there any specific guide someone can point me to do a 500->800 update? Would like to update my Aeotek 500 (ZW090) to Zooz 800 usb stick but afraid to do the plunge until someone confirms it working :)

I just did it last week - I did download the nvm backup from my Aeotec 500-series, did the nvmedit conversion manually, then restored it, though. The Zooz 800LR controller has been flawless since.

PeteRager commented 7 months ago

The Zooz 800LR controller has been flawless since.

What FW version is on the 800 LR?

skynet01 commented 7 months ago

Is there any specific guide someone can point me to do a 500->800 update? Would like to update my Aeotek 500 (ZW090) to Zooz 800 usb stick but afraid to do the plunge until someone confirms it working :)

I just did it last week - I did download the nvm backup from my Aeotec 500-series, did the nvmedit conversion manually, then restored it, though. The Zooz 800LR controller has been flawless since.

Whats this nvmedit conversion that you had to do? And i thought nvmedit is done behind the scenes now?

mkosmo commented 7 months ago

What FW version is on the 800 LR?

FW: v1.20 SDK: v7.19.3

Whats this nvmedit conversion that you had to do? And i thought nvmedit is done behind the scenes now?

I did it by hand since the version of zwavejs running in my HA environment isn't running the latest.

jum0n commented 7 months ago

I did it by hand since the version of zwavejs running in my HA environment isn't running the latest.

What version of Zwave JS is minimum for this? I see that version I'm running (included with Zwave JS UI in HA) is currently at 12.4.4. Thanks!

Scoop2389 commented 7 months ago

What version of Zwave JS is minimum for this? I see that version I'm running (included with Zwave JS UI in HA) is currently at 12.4.4. Thanks!

You need to have Z-Wave JS UI installed in order to do NVM backup and restore through that add-on's UI. Otherwise you can use the raw API as mentioned in this comment although this comes with a high risk of messing up your installation if you don't know what you're doing.

This isn't a Home Assistant specific thread (correct me if I'm wrong) so it details all the possibilities that other programs built on nodejs (which includes home assistant Z-Wave plugins) can support, but it's still up to the actual author of the application that you use (either the Z-Wave JS or Z-Wave JS UI add-ons in this case) to support it.

Using the raw API skips the plugin and talks directly to nodejs, making it possible to use features that aren't supported by the programs (because the programs are calling the API).

Hopefully this helps and I didn't just make it more confusing/am completely wrong 🙂

joaldes commented 7 months ago

What FW version is on the 800 LR?

FW: v1.20 SDK: v7.19.3

Whats this nvmedit conversion that you had to do? And i thought nvmedit is done behind the scenes now?

I did it by hand since the version of zwavejs running in my HA environment isn't running the latest.

I'm also on HA and the backup/restore didn't work for me so I wonder if this is the reason. How did you do the nvmedit manually?

gardiner4 commented 7 months ago

nvmedit is automatically done behind the scenes. As long as the 500 series is running SDK 6.61 or newer, backup+restore should just work.

I tried going from a 500->800. It completes a restore but the 800 is still empty post driver restart. I then went 500->700 and it was successful. I then tried 700->800 and it was still empty. All sticks are running the current firmware.

Hedda commented 7 months ago

This isn't a Home Assistant specific thread (correct me if I'm wrong) so it details all the possibilities that other programs built on nodejs (which includes home assistant Z-Wave plugins) can support, but it's still up to the actual author of the application that you use (either the Z-Wave JS or Z-Wave JS UI add-ons in this case) to support it.

True that this is not a Home Assistant specific thread, but the raised question is very relvant here I think seeing that Home Assistant users is the largest userbase and its developers the leaders of this project too

jum0n commented 7 months ago

Hopefully this helps and I didn't just make it more confusing/am completely wrong 🙂

That actually clears it up for me but new questions (probably not for this forum) come to mind such as how can I find out what version of nodejs is being used in my version of Zwave JS UI and what version of nodejs has this functionality. Thanks!!

chrispatin commented 7 months ago

The migration I performed was using the HA Addon ZWave JS UI v9.9.0.

Used the Backup / Restore functionality provided by the UI.

stephenjamieson commented 7 months ago

After restoring, should the 800 controller now show that it's the old 500 controller?

Screenshot 2024-02-28 at 11 16 20 AM
jeffjensen commented 7 months ago

how can I find out what version of nodejs is being used in my version of Zwave JS UI and what version of nodejs has this functionality. Thanks!!

@jum0n Bookmark this handy page: https://zwave-js.github.io/which-version

Scoop2389 commented 7 months ago

True that this is not a Home Assistant specific thread, but the raised question is very relvant here I think seeing that Home Assistant users is the largest userbase and its developers the leaders of this project too

Totally agree that this question belongs here, just trying to explain why the issue tracker may not represent what is available in home assistant, and hopefully I did alright 🙂.

mkosmo commented 7 months ago

I'm also on HA and the backup/restore didn't work for me so I wonder if this is the reason. How did you do the nvmedit manually?

Using node-zwave-js directly, v12.4.4, via npx. Downloaded the nvm backup via the HA addon, did the conversion with the node module directly, and then restored using the HA addon.

zacs commented 7 months ago

After restoring, should the 800 controller now show that it's the old 500 controller?

Screenshot 2024-02-28 at 11 16 20 AM

I did a normal backup and restore on 12.4.4 (none of the command line conversion) and it went super well, very easy. But now I realize my ZST39 800-series is saying the same thing. I'm not sure it matters, since the network seems to be doing great for the past 24hrs.

image

stephenjamieson commented 7 months ago

I re-flashed the zooz firmware and that updated to the correct info, not sure if it was necessary, but I wanted to be ready for LR when it's available.

jcustenborder commented 7 months ago

I was able to get a 700 - 800 backup and restore to work. Both with Zooz sticks. I have two installs of zwavejs. Both of them were able to do the restore and all devices came across. One of the installs is getting into a weird issue where the driver goes into a bad state requiring a restart of the container.

2024-03-07 12:15:18.349 ERROR Z-WAVE-SERVER: Failed to send the command after 5 attempts (ZW0202)
ZWaveError: Failed to send the command after 5 attempts (ZW0202)
    at Driver.sendMessage (/usr/src/app/node_modules/zwave-js/src/lib/driver/Driver.ts:5256:23)
    at Driver.sendCommandInternal (/usr/src/app/node_modules/zwave-js/src/lib/driver/Driver.ts:5454:28)
    at Driver.sendCommand (/usr/src/app/node_modules/zwave-js/src/lib/driver/Driver.ts:5565:15)
    at Proxy.set (/usr/src/app/node_modules/@zwave-js/cc/src/cc/MultilevelSwitchCC.ts:264:24)
    at Proxy.<anonymous> (/usr/src/app/node_modules/@zwave-js/cc/src/cc/MultilevelSwitchCC.ts:341:17)
    at ZWaveNode.setValue (/usr/src/app/node_modules/zwave-js/src/lib/node/Node.ts:1150:39)
    at NodeMessageHandler.handle (/usr/src/app/node_modules/@zwave-js/server/dist/lib/node/message_handler.js:19:43)
    at Object.node (/usr/src/app/node_modules/@zwave-js/server/dist/lib/server.js:42:92)
    at Client.receiveMessage (/usr/src/app/node_modules/@zwave-js/server/dist/lib/server.js:125:99)
    at WebSocket.<anonymous> (/usr/src/app/node_modules/@zwave-js/server/dist/lib/server.js:57:45)
2024-03-07 12:26:31.834 ERROR Z-WAVE-SERVER: Failed to send the command after 5 attempts (ZW0202)
ZWaveError: Failed to send the command after 5 attempts (ZW0202)
    at Driver.sendMessage (/usr/src/app/node_modules/zwave-js/src/lib/driver/Driver.ts:5256:23)
    at Driver.sendCommandInternal (/usr/src/app/node_modules/zwave-js/src/lib/driver/Driver.ts:5454:28)
    at Driver.sendCommand (/usr/src/app/node_modules/zwave-js/src/lib/driver/Driver.ts:5565:15)
    at Proxy.set (/usr/src/app/node_modules/@zwave-js/cc/src/cc/MultilevelSwitchCC.ts:264:24)
    at Proxy.<anonymous> (/usr/src/app/node_modules/@zwave-js/cc/src/cc/MultilevelSwitchCC.ts:341:17)
    at ZWaveNode.setValue (/usr/src/app/node_modules/zwave-js/src/lib/node/Node.ts:1150:39)
    at NodeMessageHandler.handle (/usr/src/app/node_modules/@zwave-js/server/dist/lib/node/message_handler.js:19:43)
    at Object.node (/usr/src/app/node_modules/@zwave-js/server/dist/lib/server.js:42:92)
    at Client.receiveMessage (/usr/src/app/node_modules/@zwave-js/server/dist/lib/server.js:125:99)
    at WebSocket.<anonymous> (/usr/src/app/node_modules/@zwave-js/server/dist/lib/server.js:57:45)
bryanklingner commented 6 months ago

Just another report of success migrating from a Aotec 500 series stick to Zooz ZST39 LR purely using the Home Assistant Z-Wave JS web UI.

  1. My old controller was an Aotec Z-Stick Gen5+. I used the Zwave JS Home Assistant UI to create an NVM backup.
  2. Shut down the HA server completely, removed the old stick.
  3. I plugged in a fresh-from-the-box Zooz 800 LR stick (ZST39 LR), came with FW: v1.20 / SDK: v7.19.3. Turned back on the HA server. The stick was automatically recognized at the same path as the old stick (no need to select the path manually.
  4. I used the Zwave JS Home Assistant UI to restore from the NVM backup made in step 1
  5. The Zwave network came back with all nodes, and with the stick reporting it was an Aotec Gen5+ as reported by others above
  6. I reflashed the stick firmware to v 1.2 using the Zwave JS Home Assistant UI, now it reports the correct stick [per notes below this was unnecessary; could have just restarted Z-Wave JS UI]
PeteRager commented 6 months ago

2. fresh-from-the-box Zooz 800 LR stick (ZST39 LR)

What firmware version was this at?

bryanklingner commented 6 months ago
  1. fresh-from-the-box Zooz 800 LR stick (ZST39 LR)

What firmware version was this at?

Edited my response, it came with latest FW: v1.20 / SDK: v7.19.3

AlCalzone commented 6 months ago

AFAIK restarting Z-Wave JS UI is enough to get the correct label to display for the controller.