search

OpenZWave / Zwave2Mqtt

Fully configurable Zwave to MQTT gateway and Control Panel using NodeJS and Vue
MIT License
356 stars 92 forks source link

[bug] BitSet value types are missing #90

Closed sjorge closed 4 years ago

sjorge commented 5 years ago

Version

Build/Run method

Zwave2Mqtt version: 2.0.2-master @ 20190809 Openzwave Version: 1.6

Describe the bug I noticed a parameter that use to be able to be configure is now missing, this is becasue the field type changed to bitset from int (as it is indeed a bitset).

To Reproduce Look for a node with a bitset parameter.

Expected behavior The field to show up and be configurable.

Additional context Example device is a ZW100, here is a snippit from the config for one of the missing params:

    <Value bitmask="241" genre="config" index="101" label="Group 1 Reports" size="4" type="bitset" value="241">
      <Help>Which reports need to send automatically in timing intervals for group 1.
                                Bitmask:
                                10000000 - 128 - Luminance
                                01000000 - 64 - Humidity
                                00100000 - 32 - Temperature
                                00010000 - 16 - Ultraviolet
                                00000001 - 1 - Battery
                                10100000 - 160 - Luminance and Temperature (32+128)
                                11110001 - 241 - All (1+16+32+64+128)
                        </Help>
      <BitSet id="1">
        <Label>Battery</Label>
        <Help>Send Battery Report</Help>
      </BitSet>
      <BitSet id="5">
        <Label>Ultraviolet</Label>
        <Help>Send Ultraviolet Report</Help>
      </BitSet>
      <BitSet id="6">
        <Label>Temperature</Label>
        <Help>Send Temperature Report</Help>
      </BitSet>
      <BitSet id="7">
        <Label>Humidity</Label>
        <Help>Send Humidity Report</Help>
      </BitSet>
      <BitSet id="8">
        <Label>Luminance</Label>
        <Help>Send Luminance Report</Help>
      </BitSet>
    </Value>
sjorge commented 5 years ago

They show up again as a normal integer, so no selection (and with the updated configs, the hits on what to set is gone too as those are now encoded in the BitSet blocks)

robertsLando commented 5 years ago

But there was an error, segmentation fault, is it gone? Could you also add a debugger in chrome in the socket INIT to check the content of the valueid of bitset to check if now it has a property named bitIds (check pr for more info). Just hit F12 and set go to sources tab, webpack open ControlPanel.vue and check for the socket ‘INIT’ there you wull get all nodes

Daniel

On 27 Sep 2019, at 19:36, Jorge Schrauwen notifications@github.com wrote:

They show up again as a normal integer, so no selection (and with the updated configs, the hits on what to set is gone too as those are now encoded in the BitSet blocks)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

robertsLando commented 5 years ago

Here: https://github.com/OpenZWave/Zwave2Mqtt/blob/master/src/components/ControlPanel.vue#L1079

Daniel

On 27 Sep 2019, at 19:36, Jorge Schrauwen notifications@github.com wrote:

They show up again as a normal integer, so no selection (and with the updated configs, the hits on what to set is gone too as those are now encoded in the BitSet blocks)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

sjorge commented 5 years ago

Wait, I might have forgot to do the seperate npm install for the update node-openzwave-shared. So this was with the normal release of node bindings... I'll try again later.

sjorge commented 5 years ago 
npm install OpenZWave/node-openzwave-shared#feature/BitSet
npm run build

As I was already at master for zwave2mqtt, it seems to start up without a segfault now. Will check the interface in a bit, once the network has stabelized because it takes ages :(

[root@amethyst /opt/zwave2mqtt]# git diff | grep openzwave
     "openzwave-shared": {
-      "resolved": "https://registry.npmjs.org/openzwave-shared/-/openzwave-shared-1.5.6.tgz",
+      "version": "github:OpenZWave/node-openzwave-shared#b7672311b574cf2ffadba672960652fc2c918752",
+      "from": "github:OpenZWave/node-openzwave-shared#feature/BitSet",
-    "openzwave-shared": "^1.5.6",
+    "openzwave-shared": "github:OpenZWave/node-openzwave-shared#feature/BitSet",

No segfault this time! Still just shows up as a normal int in the webinterface, will try the devloper thingy

sjorge commented 5 years ago

Imgur

sjorge commented 5 years ago

Ah good news! if I hover on 'data' I can drill down to one of the BitSet values! So the data is now getting passed from correctly, just not displayed yet... but this is a awesome first step.

Imgur

robertsLando commented 5 years ago

That’s wonderful! Just need to wait @ekrak to make a new release I will also start to implement the support in control panel ui to show and send bitset values in a user friendly way. Really really thanks @sjorge for the quick response 😊

Daniel

On 27 Sep 2019, at 20:32, Jorge Schrauwen notifications@github.com wrote:

Ah good news! if I hover on 'data' I can drill down to one of the BitSet values!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

sjorge commented 5 years ago

❤️ I'll keep testing for you once we're at the next step

robertsLando commented 5 years ago

@sjorge Could you please paste here all the valueid json of bitset valueid please?

(all the content showed inside 112-1-48 in the screenshot above)

sjorge commented 5 years ago

How do I copy those? I did not find a way to copy them, hens the screenshots

robertsLando commented 5 years ago

Just add the breakpoint in the same line than once the breakpoint is hitted type in chrome console (it should be at the bottom) data.nodes[nodeindex].values['112-1-48'] than press enter end it will be showed in console

robertsLando commented 5 years ago

I have started implementing the management on server and ui sides but I need something to test them then I can release it for you to test if its working with phisical devices :)

robertsLando commented 5 years ago

@sjorge Try this dc3e6ae94d6a2a95049f5a115b65ec00667c536c

sjorge commented 5 years ago

I'm not having any success dumping the values, it keeps saying data (or nodes) is undefiend in the console. I'll try to pull in the new commit in a bit.

sjorge commented 5 years ago

Currently it looks like: Imgur

sjorge commented 5 years ago

It doesn't seem to parse the current value (or actually not read it form the device?) though.

robertsLando commented 5 years ago

Try with latest commit

Daniel

On 1 Oct 2019, at 15:18, Jorge Schrauwen notifications@github.com wrote:

It doesn't seem to parse the current value (or actually not read it form the device?) though.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

sjorge commented 5 years ago

Nope, but I think this might be hte other bug where the value-id lookup doen at startup timeout for devices that are sleeping. 🤔

sjorge commented 5 years ago

I did notice on some values not all are listed...

5-112-1-101

Imgur

The XML and object list the same bitset fields for it:

                                <Value type="bitset" genre="config" instance="1" index="101" label="Group 1 Reports" units="" read_only="false" write_only="false" verify_changes="false" poll_intensity="0" min="0" max
="0" bitmask="241" value="241" size="4">
                                        <Help>Which reports need to send automatically in timing intervals for group 1. Bitmask: 10000000 - 128 - Luminance 01000000 - 64 - Humidity 00100000 - 32 - Temperature 0001000
0 - 16 - Ultraviolet 00000001 - 1 - Battery 10100000 - 160 - Luminance and Temperature (32+128) 11110001 - 241 - All (1+16+32+64+128)</Help>
                                        <BitSet id="1">
                                                <Label>Battery</Label>
                                                <Help>Send Battery Report</Help>
                                        </BitSet>
                                        <BitSet id="5">
                                                <Label>Ultraviolet</Label>
                                                <Help>Send Ultraviolet Report</Help>
                                        </BitSet>
                                        <BitSet id="6">
                                                <Label>Temperature</Label>
                                                <Help>Send Temperature Report</Help>
                                        </BitSet>
                                        <BitSet id="7">
                                                <Label>Humidity</Label>
                                                <Help>Send Humidity Report</Help>
                                        </BitSet>
                                        <BitSet id="8">
                                                <Label>Luminance</Label>
                                                <Help>Send Luminance Report</Help>
                                        </BitSet>
                                </Value>

Also only the config ID is show, not the label: in this case label="Group 1 Reports"

robertsLando commented 5 years ago

@sjorge I need you to paste me the json of the entire valueid bitset type to make some tests :( provide more than one if you have more

robertsLando commented 5 years ago

I have added label to bitset values in latest commit. Now I need to understand others problems

sjorge commented 5 years ago

Layout is a bit messed up, good news though the one showing the wrong result is probably due to a openzwave bug, the cache has the correct bitset fields but the data.nodes entry has what you are displaying :(

Imgur

The help value should probably be a tooltip for the actual toggle+label?

Hopefully 1.6 will get better over time.

I had no luck getting the full json, I can only explorer it by hovering but using the console what at the breakpoint just says data or nodes is undefined. I assume because the breakpoint is inside an event?

I was able to dump the whole xhr request though, it looks to contain what you need and more.


 "112-1-101": {
          "value_id": "5-112-1-101",
          "node_id": 5,
          "class_id": 112,
          "type": "bitset",
          "genre": "config",
          "instance": 1,
          "index": 101,
          "label": "Group 1 Reports",
          "units": "",
          "help": "Which reports need to send automatically in timing intervals for group 1. Bitmask: 10000000 - 128 - Luminance 01000000 - 64 - Humidity 00100000 - 32 - Temperature 00010000 - 16 - Ultraviolet 00000001 - 1 - Battery 10100000 - 160 - Luminance and Temperature (32+128) 11110001 - 241 - All (1+16+32+64+128)",
          "read_only": false,
          "write_only": false,
          "min": 0,
          "max": 0,
          "is_polled": false,
          "bitSetIds": {
            "1": {
              "help": "Send Battery Report",
              "label": "Battery",
              "value": false
            },
            "3": {
              "help": "",
              "label": "Reserved",
              "value": false
            },
            "4": {
              "help": "",
              "label": "Reserved",
              "value": false
            },
            "5": {
              "help": "Send Ultraviolet Report",
              "label": "Ultraviolet",
              "value": false
            }
          },
          "bitMask": 241,
          "value": 226
        },

 "112-1-48": {
          "value_id": "5-112-1-48",
          "node_id": 5,
          "class_id": 112,
          "type": "bitset",
          "genre": "config",
          "instance": 1,
          "index": 48,
          "label": "Enable/disable to send a report on Threshold",
          "units": "",
          "help": "Enable/disable to send a report when the measurement is more than the upper limit value or less than the lower limit value. Note: If USB power, the Sensor will check the limit every 10 seconds. If battery power, the Sensor will check the limit when it is waken up.",
          "read_only": false,
          "write_only": false,
          "min": 0,
          "max": 0,
          "is_polled": false,
          "bitSetIds": {
            "1": {
              "help": "Lower Temperature Threshold",
              "label": "Lower Temperature",
              "value": false
            },
            "2": {
              "help": "Lower Humdity Threshold",
              "label": "Lower Humidity",
              "value": false
            },
            "3": {
              "help": "Lower Luminance Threshold",
              "label": "Lower Luminance",
              "value": false
            },
            "4": {
              "help": "Lower Ultraviolet Threshold",
              "label": "Lower Ultraviolet",
              "value": false
            },
            "5": {
              "help": "Upper Temerature Threshold",
              "label": "Upper Temperature",
              "value": false
            },
            "6": {
              "help": "Upper Humdity Threshold",
              "label": "Upper Humidity",
              "value": false
            },
            "7": {
              "help": "Upper Luminance Threshold",
              "label": "Upper Luminance",
              "value": false
            },
            "8": {
              "help": "Upper Ultraviolet Threshold",
              "label": "Upper Ultraviolet",
              "value": false
            }
          },
          "bitMask": 255,
          "value": 0
        }
917huB commented 5 years ago

Upgraded to latest this morning and wanted to share my findings to hopefully try and help.

zw100

i) The bitmask was extended from a 4 to 8 bit integer in a recent version so this doesnt appear to display those higher order bits. ii) The binary settings only appear to group 1 (112-1-101) and this switch has 3 groups (112-1-102, 112-1-103)to allow for different sensor reporting and frequency.

My screengrab is attached illustrating what I'm seeing. The bitmask values are in the helper text too.

Also, heres the engineering sheet for firmware, 1.11 which details bitmasks etc.

robertsLando commented 5 years ago

Did you try to install the latest version from master? It has ui improvments for bitmask values

Daniel

On 20 Oct 2019, at 22:23, 917huB notifications@github.com wrote:

 Upgraded to latest this morning and wanted to share my findings to hopefully try and help.

i) The bitmask was extended from a 4 to 8 bit integer in a recent version so this doesnt appear to display those higher order bits. ii) The binary settings only appear to group 1 (112-1-101) and this switch has 3 groups (112-1-102, 112-1-103)to allow for different sensor reporting and frequency.

My screengrab is attached illustrating what I'm seeing. The bitmask values are in the helper text too.

Also, heres the engineering sheet for firmware, 1.11 which details bitmasks etc.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

917huB commented 5 years ago

this was from a sync with master. Previously I didn't see these toggles so I think my install is at master and correctly configured. Im new to node/linux though possible I could have missed something.

robertsLando commented 5 years ago

You also have to run “npm install” or better “npm install openzwave-shared@1.5.8” because it has been updated

Daniel

On 20 Oct 2019, at 22:42, 917huB notifications@github.com wrote:

 this was from a sync with master. Previously I didn't see these toggles so I think my install is at master and correctly configured. Im new to node/linux though possible I could have missed something.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

917huB commented 5 years ago

To the best of my knowledge I'm now running latest across my zwave stack and still see the same behaviour I reported above, i.e namely incomplete bitfields.

For clarification, I am now running with the following versions pm2: 3.5.1 npm 6.12.0 OpenZWave: 1.6.945 openzwave-shared@1.5.8 root@homeassistant:/opt/Zwave2Mqtt# git log -1 commit 85b49dcf00c3efdb3825e42d93b265052a9f58a6

Either I'm missing something in my install, or maybe my screenshot wasn't clear. I'd like to try and verify what I'm seeing as this update is going to make it impossible for me to update this setting now we don't have the ability to enter integers directly.

For field "112-1-48", I correctly see the 8 toggles as shown by sjorge above for send report on upper.lower thresholds correctly.

My problem is with field "112-1-101" which displays four toggles, Battery, reserved, reserved, ultraviolet when it should display eight (from page 16 in the manual I linked in my original message above) Luminance, Humidity, Temperature, Ultraviolet, Reserved, Reserved, Reserved, Battery

fields "112-1-102" and "112-1-103" should also display the same fields too.

If this is still a case of user error, I apologise for wasting your time.

robertsLando commented 5 years ago

Hi @917huB thanks for your feedback! Unfortunally I have no way to test this feature, I can only ask you if you can paste here the zwcfg_<homeid>.xml file that you can find inside store folder, don't paste the entire file content just the part about the bitset values to check if there the missing bitset are showing. Than I ask to @Fishwaldo if this is a known bug or not, if not I think the error must be in this lines of code: https://github.com/OpenZWave/node-openzwave-shared/blob/master/src/utils.cc#L175

This is the loop I use to get the valid bitmask values and popolate bitsetids object. Please give it a check and tell me if you see something wrong

917huB commented 5 years ago

I didn't have a zwcfg.xml but did have a ozwcache.xml - I'll need to figure out why it isnt saving. Is this what you are after though? I think there is a bug in this definition file because index 102 & 103 should be identical to 101 and they aren't. Index 101 appears to have the correct bit fields set, but isnt matching what's being displayed in my Zwave2mqtt panel. 

   <Value type="bitset" genre="config" instance="1" index="101" label="Group 1 Reports" units="" read_only="false" write_only="false" verify_changes="false" poll_intensity="0" min="0" max="0" bitmask="241" value="241" size="4">
                                        <Help>Which reports need to send automatically in timing intervals for group 1. Bitmask: 10000000 - 128 - Luminance 01000000 - 64 - Humidity 00100000 - 32 - Temperature 00010000 - 16 - Ultraviolet 00000001 - 1 - Battery 10100000 - 160 - Luminance and Temperature (32+128) 11110001 - 241 - All (1+16+32+64+128)</Help>
                                        <BitSet id="1">
                                                <Label>Battery</Label>
                                                <Help>Send Battery Report</Help>
                                        </BitSet>
                                        <BitSet id="5">
                                                <Label>Ultraviolet</Label>
                                                <Help>Send Ultraviolet Report</Help>
                                        </BitSet>
                                        <BitSet id="6">
                                                <Label>Temperature</Label>
                                                <Help>Send Temperature Report</Help>
                                        </BitSet>
                                        <BitSet id="7">
                                                <Label>Humidity</Label>
                                                <Help>Send Humidity Report</Help>
                                        </BitSet>
                                        <BitSet id="8">
                                                <Label>Luminance</Label>
                                                <Help>Send Luminance Report</Help>
                                        </BitSet>
                                </Value>
                                <Value type="int" genre="config" instance="1" index="102" label="Group 2 Reports" units="" read_only="false" write_only="false" verify_changes="false" poll_intensity="0" min="-2147483648" max="2147483647" value="0">
                                        <Help>Which reports need to send automatically in timing intervals for group 2. Bitmask: 10000000 - 128 - Luminance 01000000 - 64 - Humidity 00100000 - 32 - Temperature 00010000 - 16 - Ultraviolet 00000001 - 1 - Battery 10100000 - 160 - Luminance and Temperature (32+128) 11110001 - 241 - All (1+16+32+64+128)</Help>
                                </Value>
                                <Value type="int" genre="config" instance="1" index="103" label="Group 3 Reports" units="" read_only="false" write_only="false" verify_changes="false" poll_intensity="0" min="-2147483648" max="2147483647" value="0">
                                        <Help>Which reports need to send automatically in timing intervals for group 3. Bitmask: 10000000 - 128 - Luminance 01000000 - 64 - Humidity 00100000 - 32 - Temperature 00010000 - 16 - Ultraviolet 00000001 - 1 - Battery 10100000 - 160 - Luminance and Temperature (32+128) 11110001 - 241 - All (1+16+32+64+128)</Help>
                                </Value>
robertsLando commented 5 years ago

Ok so I think it is most likely a bug on Openzwave side

Daniel

On 22 Oct 2019, at 09:13, 917huB notifications@github.com wrote:

 I didn't have a zwcfg.xml but did have a ozwcache.xml - I'll need to figure out why it isnt saving. Is this what you are after though? I think there is a bug in this definition file because index 102 & 103 should be identical to 101 and they aren't. Index 101 appears to have the correct bit fields set, but isnt matching what's being displayed in my Zwave2mqtt panel.

Which reports need to send automatically in timing intervals for group 1. Bitmask: 10000000 - 128 - Luminance 01000000 - 64 - Humidity 00100000 - 32 - Temperature 00010000 - 16 - Ultraviolet 00000001 - 1 - Battery 10100000 - 160 - Luminance and Temperature (32+128) 11110001 - 241 - All (1+16+32+64+128) Send Battery Report Send Ultraviolet Report Send Temperature Report Send Humidity Report Send Luminance Report Which reports need to send automatically in timing intervals for group 2. Bitmask: 10000000 - 128 - Luminance 01000000 - 64 - Humidity 00100000 - 32 - Temperature 00010000 - 16 - Ultraviolet 00000001 - 1 - Battery 10100000 - 160 - Luminance and Temperature (32+128) 11110001 - 241 - All (1+16+32+64+128) Which reports need to send automatically in timing intervals for group 3. Bitmask: 10000000 - 128 - Luminance 01000000 - 64 - Humidity 00100000 - 32 - Temperature 00010000 - 16 - Ultraviolet 00000001 - 1 - Battery 10100000 - 160 - Luminance and Temperature (32+128) 11110001 - 241 - All (1+16+32+64+128)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

Fishwaldo commented 5 years ago

Not a bug - Just the "migration" of config files to BitSet is stalled. This device just had one update.

PR's welcome to update the config file.

Fishwaldo commented 5 years ago

Index 101 appears to have the correct bit fields set, but isnt matching what's being displayed in my Zwave2mqtt panel.

thats something to investigate tho. 102 and 103 are not setup as BitSet in the config file

robertsLando commented 5 years ago

So @sjorge @917huB in poor words you should go here find your device xml configuration and fix it by adding missing bitset values and submit a PR :)

robertsLando commented 5 years ago

@sjorge any news?