[feat] Provide a database of known working, "supported" devices

[blhoward2]

Is your feature request related to a problem? Please describe.

As it stands, people frequently don't know whether a device is supported until after they purchase it. This leads to frustration and increased pressure on the project, as users press for rapid changes to incorporate new devices. Additionally, it is difficult for someone in the market for, say switches or water sensors, to determine which among the many models are already supported if they do not wish to wait for a newly added device. (Credit to my wife for thinking of this...) The goal would be to reduce pressure for immediate changes as the project grows, as well as to put users on notice if you will that the device they are buying is not yet supported and may not work out of the box.

This might more properly live under node-zwave-js, but people aren't likely to know to look there to determine support by zwavejs2mqtt.

Describe the solution you'd like

Build a table of devices by device type, then brand and model, breaking down whether a device driver exists, whether the device is known to work well, or documenting specific challenges with a device. For some, it may be helpful to allow specific instructions wiki style. (E.g. for the Leviton vrcs4 scene controllers you need to download software for your zwave stick to change the device settings in the firmware so it sends scene events instead of node events for the best functionality, at least under ozw. Finding these instructions requires a fair bit of googling or digging deep through HA Community topics.)

Additionally, that would be a natural place to outline instructions for how people should submit new device files. E.g. how to do a PR, how to submit it, etc.

[robertsLando]

@AlCalzone Thoughts? I think it could be added to docs, but should be keep updated by users

[AlCalzone]

I thought about this shortly yesterday. What we could do is auto-generate an overview from the config files and use a flag in there to tell if it was verified by users that the device works with zwave-js. However we'd need a comfortable way for users to say "this device works", or that will never be up to date. Maybe some feedback button in the UI? Or we need something like OpenHAB does: https://opensmarthouse.org/zwavedatabase/ if someone is willing to go the distance. That could even be coupled with Github for automatic PRs after one edits the DB there...

[robertsLando]

I'm +1 about adding a flag like tested that marks a device as tested. Then we should create something like the openhub table you linked that will be auto generated by reading config files

[marcus-j-davies]

So - I want to push this, being heavily involved with web services development in my professional life, I'm quite eager.

just a few points come to mind.

• have it as an additional page on Docs, or have it as a repo (we could link to/from), I Don't have permission on the main project, lets keep it that way ;-)
• Upon reading into Pages - it doesn't support server side scripts, and raising PR's to update the flag on the config package via the HTTP REST API, wont work without authorisation in the header of the HTTP request, which then opens up a security issue, as it can be inspected by the end user without issue.
• Initially, I saw this as an exact mirror of the config package (without the ability for user to update) but keep it updated by periodically taking a snapshot of the config package. If we want point 2 to happen - we need to to somehow overcome that obstacle (i.e hosting away from pages - where we can use server side code)
  • Should this issue be transferred?
  • What did I miss?

[blhoward2]

Those are good questions. I’m in the process of processing all of the zwave alliance xml files, which will vastly expand the number of devices covered (as many as 2,000 additional devices, though it may be less once I combine variations of models into one). The xmls include within them inclusion/exclusion instructions for most, how to reset, etc. I’m not sure all of that will continue to live in the device_config file, so this may need to be maintained in a separate database. Framework work is fine but I’d wait to actually handjam any documentation as the goal will be to populate as much as can be done programmatically.

The ultimate location is an interesting question. The device files belong to node-zwave-us but end users will come here (and to the new HA zwave-js-server), and other projects use node-zwave-us, so perhaps the reference should be included in the documentation for all three.

Does anyone have a server available that can do server-side that could be linked to? If not, I’d potentially be willing to spring for one at somewhere like bluehost.

[marcus-j-davies]

I do have a host - no console access, but I do have a DB/PHP available, we could use it as a testing platform?

[marcus-j-davies]

or I can put my Pi to work :smile: - again for testing

[blhoward2]

At least for testing that may help. (I have a local server for testing too but don’t want to use in production.) I’ll defer to your expertise but I’m assuming given the sheer number of devices that serving this dynamically may make more sense than a series of static webpages that get updated? (Though that could be done by pulling the repo and having the import config process include adding them to the static page.)

[blhoward2]

I find this ugly and hard to use: http://openzwave.com/device-database

This is also hard to navigate, but this is potentially the scope of devices we’re talking about: https://products.z-wavealliance.org/

One trick is that you want people to be able to confirm if a specific device is supported, but you also want to be able to show people all water sensors supported in case they’re looking to buy one (I think), etc.

[blhoward2]

Of course, the projects belong to @robertsLando and @AlCalzone, so their thoughts carry more weight, lol.

[marcus-j-davies]

it could be static pages (as apposed to the pages being rendered behind the scene) given the configs are in JSON, javascript could render the result client side.

The biggest hurdle is handling 'This Device Works', without leaking security information, as needed to raise PR's if that isn't a must. - we could do all this on pages, if it is a must, we need server side.

Of course, the projects belong to robertsLando and AlCalzone, so their thoughts carry more weight, lol.

but yes, we'll see where this goes.

[AlCalzone]

Just a thought: I'm currently experimenting with Heroku for the repo bot where you can deploy node.js apps almost plug and play (Auto-sync/deploy from github). Maybe that is something we can consider.

[blhoward2]

One constraint is that with this many devices, there is a risk that a user could mess up a device file by modifying something if they have the ability to edit directions or anything wiki style and save back to the device file, even if in a supplementary field. Yet it may be a lot to keep up if people can’t change things. There would have to be a lot of error checking and such to ensure that anything saved back to the config file was compatible with the json loader. That’s why unless this will be strictly a list of supported devices (which maybe that is enough), I think it would have to live on its own and just be populated from the config files at setup and on subsequent imports.

I guess it all depends on the scope of what the final product will include.

[marcus-j-davies]

@AlCalzone

Just a thought: I'm currently experimenting with Heroku for the repo bot where you can deploy node.js apps almost plug and play (Auto-sync/deploy from github). Maybe that is something we can consider.

Potentially, if these Heroku apps can host a HTTP port (allowing us to serve content), we could plug the backend of it (by way of consuming post requests) into the Github API, where all the authentication, is done 'of camera' rasing the PR.

I would want to incorporate captcha with any solution we come up with.

EDIT: we wouldn't Evan need to sever the content with it just a way to pass something to it VIA HTTP, without the need for auth. but the Heroku app itself does the auth part with GitHub, resulting in a PR.

The biggest hurdle is being able to raise a PR - but without given away security information, as required by Githubs API. if we can pass an unauthenticated request to something that can raise the PR (where it has the rights to raise a PR) - we're golden, raising the PR using the HTML/javascript alone will require that it has access to tokens and such, so doing it purely on the users browser is unsafe, we need some kind of 'middleware' to raise the PR, and where the security info is 'off camera'

this is all in light that we want the ability for users to flag working devices.

[marcus-j-davies]

@blhoward2

I think it would have to live on its own and just be populated from the config files at setup and on subsequent imports.

I am actually in favour of having this as more of a pretty mirror for the config package (that can be easily updated with a schedule), allowing a reference for users.

we could later - look at allowing it to have 'This Device Works' functionality. but will need to overcome the hurdles I mentioned.

[blhoward2]

I defer to your experience on the best roadmap. I have no idea what it would take to transition to that versus building it with that view in mind to start.

[AlCalzone]

The biggest hurdle is being able to raise a PR - but without given away security information, as required by Githubs API. if we can pass an unauthenticated request to something that can raise the PR (where it has the rights to raise a PR) - we're golden, raising the PR using the HTML/javascript alone will require that it has access to tokens and such, so doing it purely on the users browser is unsafe, we need some kind of 'middleware' to raise the PR, and where the security info is 'off camera'

On Heroku you have the ability to define environment variables (e.g. for authentication). The same should be true for other hosters (don't want to advocate for Heroku here), but I just found it very easy to work with, after having dabbled with Express on AWS Lambda and Glitch.

So that would probably work, but I'm not sure how to stop people from misusing/spamming that API. Maybe authenticate with GitHub accounts after all?

[marcus-j-davies]

On Heroku you have the ability to define environment variables (e.g. for authentication).

Does Heroku allow for serving up http ports?

if so, we could simply create an express app, allow it to parse the config package, creating the searchable database as such. then on the event of someone clicking 'this works' process the PR etc etc using Gits REST API, this gives us our 'of camera'

Maybe authenticate with GitHub accounts after all?

we could use the oauth provider they have, I haven't used it myself, but have used websites that do. this should add our spam filtering, we could also identify the GitHub user who 'verfiied' the device this way. I'll do some research

it all still needs thought - but is heading In the right direction.

[AlCalzone]

Does Heroku allow for serving up http ports?

Yep, that's probably one of the main use cases.

[robertsLando]

https://devices.zwave-js.io/

[AlCalzone]

Still work in progress, but yup - that's the one.