Add ability to fetch all cached values (Unmanaged)

[marcus-j-davies]

Given that getValue does not actually cause network traffic, we could return the cache, allowing recipients to configure the environment on system start up.

[crxporter]

I've been watching my devices for a couple of days now and it seems like I almost never get battery updates. In fact, only one device ever says anything about its battery - about once a day.

I'm excited for the getValue option so I can regularly poll the information which isn't updated (specifically battery)

[marcus-j-davies]

Hey,

There will be 2 new methods.

GetCache: should be used to restore last known values, on start up (still deciding if this should be for the entire network, or a specific device) PollValue: which will actually ask the device for an update.

So after these 2, you will have. GetValue: as it is now (the cached value) SetValue: as it is now GetCache: All cached Values (for system start ups) PollValue: Forces the value to be refreshed from the device.

[crxporter]

Seems like for a battery device the PollValue would be quite show to respond, right? It wouldn't be until the next time it wakes up.

GetCache should be great.

[marcus-j-davies]

PollValue would be quite show to respond, right?

For battery operated devices, Correct! The idea for PollValue is for those values that are critical to be accurate after a start up or recovering from system down time, where the cached value maybe too out of date.

GetCache: Non-Critical PollValue (Single property): Critical

The Cache is obviously kept upto date whilst the driver is running (and devices are sending updates)

[marcus-j-davies]

@crxporter

Would something like this work? This would be the result of calling GetValueDB (no longer called GetCache)

GetValueDB accepts an optional list of node IDs - params:[34,5,2] if not provided will return for EVERY node - which may be overkill for a large number of nodes, but the choice is there.

This new method, can be used at start up. where as PollValue (Single Node & Value ID) can be used to force an update on critical values, after a start up

[marcus-j-davies]

Im thinking I should rename currentValue to response so it takes on the same format seen when using GetValue for a single ValueID

[crxporter]

Whatever happens I"ll need to write a function node to deal with it - since I doubt the best option is to send everything as "VALUE_UPDATE" as I'm watching for now.

Initial states (getting the VALUE_DB) will be great as long as I have the node number, endpoint, command class, and current value. Bonus points if we can include the node name and node location in there.

I imagine the object something like:

[{"nodeName": "Porch", "nodeLocation": "Front", "valueId":{"big":"object"}, "currentValue": false}]

[crxporter]

Wait I just saw it. I was looking at it wrong. Didn't realize that level was all the nodes.

Your layout looks great, just if you can add the nodeName and nodeLocation?

[marcus-j-davies]

Adding the Node Name and location will be fine.

The object component will be an array of nodes (each with id, name, location) each node element will also have an array of values (each having a currenValue, and the valueId)

[crxporter]

Sounds great - I've just added a status catcher to my controller node, I'll probably run GetValueDB after each "All Nodes Ready!" to be sure the rest of NR knows what's what.

[marcus-j-davies]

Nice!

The result of GetValueDB....

You can use the split node to split the object into separate messages for better processing. you can then attach the split node to a switch node, and branch off based on nodeId

All the building blocks are there, I will hopefully get an update out in a couple of days.

[marcus-j-davies]

Just tested.

The split node connected to a switch node, should work well.

[crxporter]

55 values on node 9? What is that thing?

[marcus-j-davies]

Rings Keypad. quite a few config params

[crxporter]

So I guess that's not that much. My Aeotec 6 in 1 motion sensor has 79... Probably my most. I had a yale lock once with probably over 500 - it had spaces for 256 user codes and a couple settings per code. Not to mention the state stuff.

I didn't keep that device, it took like 15 minutes to interview that thing alone.

[marcus-j-davies]

Crikey!!

Hence the ability to filter to specific nodes on the GetValueDB method 😅

[marcus-j-davies]

@crxporter,

3.4.0 is out!

Deprecation Warnings

• Added a PollValue method to the Unmanaged class
• Added a GetValueDB method to the Driver class
• Added a GetNodeNeighbors method to the Controller class
• The neighbors property for each node in NODE_LIST is now set for removal. (4.0.0)
• Added Ability to adjust how frequently the values and metadata should be written to disk.
• Added Ability to set a custom location for device config files
• Node property sections no longer auto expanded in the UI (this can be changed)

• Bump Z-Wave JS

  PollValue example. This will cause the normal VALUE_UPDATED event

let MyValueID = {...}
let Message = {
    payload:{
        class: "Unmanaged",
        operation: "PollValue",
        node: 5,
        params: [MyValueID]
    }
}
return Message;

GetValueDB example.

let Message = {
    payload:{
        class: "Driver",
        operation: "GetValueDB",
        params: [3,7,24] /* Specifying specific node ID's is Optional (but recommended to begin with) */
    }
}
return Message;

as always - I'll leave open until you have had a chance to use the new method(s). the method you are likely interested in is GetValueDB (for node-red boot ups)

Have fun

[crxporter]

This is so cool. I need to build a startup function - but all of the information is there and will be easy enough to work with.

It's ok that I get excited about massive JSON arrays, right? That's not weird?

[marcus-j-davies]

It's ok that I get excited about massive JSON arrays, right? That's not weird?

absolutley fine, kind of what I am like with flow groups 🤣

I'm glad it works for you, although just remember, its from a cache, so if your system has been down for quite sometime, it maybe to far out of date for your liking.

But then thats where PollValue comes in, to forcibly refresh selective values. but of course, the cache is updated, when your devices change state anyway.

[crxporter]

Works great. Feel free to close this.

[crxporter]

Wanted to come back and say this is seriously an awesome thing to have. I've set things so whenever I get a green light "All Nodes Ready!" then I can pull the DB and push data to all of my homekit nodes all over the place.

My function is basic but gets the job done. And it should only run once every few weeks so there's not much need to make it all optamized and pretty.

const inValues = msg.payload.values;
const Name = msg.payload.nodeName;

let outputs = [];

inValues.forEach(value => {
    if (value.valueId.commandClass === 128 && value.valueId.property == "isLow") {
        outputs.push({
            "topic": Name,
            "payload": {
                StatusLowBattery: + value.currentValue,
                ChargingState: 2
            }
        })
    }
    if (value.valueId.commandClass === 128 && value.valueId.property == "level") {
        outputs.push({
            "topic": Name,
            "payload": {
                BatteryLevel: value.currentValue,
                ChargingState: 2
            }
        })
    }
    if (value.valueId.commandClass === 113 && value.valueId.propertyKey == "Door state") {
        let doorState = 23-value.currentValue;
        if (isNaN(doorState)) {
            doorState = 0;
        }
        outputs.push({
            "topic": Name,
            "payload": {
                ContactSensorState: doorState
            }
        })
    }
    if (value.valueId.commandClass === 113 && value.valueId.propertyKey == "Motion sensor status") {
        outputs.push({
            "topic": Name,
            "payload": {
                MotionDetected: Boolean(value.currentValue)
            }
        })
    }
    if (value.valueId.commandClass === 113 && value.valueId.propertyKey == "Sensor status") {
        if (value.valueId.property == "CO Alarm") {
            outputs.push({
                "topic": Name,
                "payload": {
                    CarbonMonoxideDetected: value.currentValue ? 1 : 0
                }
            })
        }
        if (value.valueId.property == "Smoke Alarm") {
            outputs.push({
                "topic": Name,
                "payload": {
                    SmokeDetected: value.currentValue ? 1 : 0
                }
            })
        }

    }
    if (value.valueId.commandClass === 49 && value.valueId.property == "Air temperature") {
        outputs.push({
            "topic": Name,
            "payload": {
                CurrentTemperature: value.currentValue
            }
        })
    }
    if (value.valueId.commandClass === 49 && value.valueId.property == "Humidity") {
        outputs.push({
            "topic": Name,
            "payload": {
                CurrentRelativeHumidity: value.currentValue
            }
        })
    }
    if (value.valueId.commandClass === 49 && value.valueId.property == "Illuminance") {
        outputs.push({
            "topic": Name,
            "payload": {
                CurrentAmbientLightLevel: value.currentValue
            }
        })
    }
});

return [outputs];

[marcus-j-davies]

Nice!

Easy to add other entries later down the line as well. Are you filtering to specific nodes on the GetValueDB call - I was curious how such a big payload would behave.

[crxporter]

Before this function I'm splitting the msg.payload.object array into individual messages per device. So this function gets hit with one message per device.

Then as you see it matches the values based on the if statements and spits out a whole bunch of outputs. After sending the GetValueDB I get all the properly formatted stuff within maybe 2 seconds?

I have 3 of these functions though, one for on/off switches, one for dimmers, and one for sensors. I'm using the location property to sort so I don't send every device to all 3 functions. The switch and dimmer ones set up my global context storage for each device.

Yes I'm pulling the entire DB, not filtering. It doesn't seem like a struggle for the pi 4, especially since it'll run so infrequently.

[crxporter]

If you wanna see the whole thing:

[{"id":"c76203e0.6e5858","type":"status","z":"65c335e0.d71ce4","name":"","scope":["c4162781.92a34"],"x":140,"y":400,"wires":[["f88b1f2b.ed12d"]]},{"id":"31e96468.0d4d7c","type":"split","z":"65c335e0.d71ce4","name":"Split","splt":"\\n","spltType":"str","arraySplt":1,"arraySpltType":"len","stream":false,"addname":"","x":330,"y":460,"wires":[["776153aa.4c97fc"]]},{"id":"28dbecc1.f11fac","type":"change","z":"65c335e0.d71ce4","name":"Move","rules":[{"t":"move","p":"payload.object","pt":"msg","to":"payload","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":190,"y":460,"wires":[["31e96468.0d4d7c"]]},{"id":"776153aa.4c97fc","type":"switch","z":"65c335e0.d71ce4","name":"Sort","property":"payload.nodeLocation","propertyType":"msg","rules":[{"t":"eq","v":"Dimmer","vt":"str"},{"t":"cont","v":"Switch","vt":"str"},{"t":"cont","v":"Sensor","vt":"str"}],"checkall":"true","repair":false,"outputs":3,"x":190,"y":540,"wires":[["2abdc0d7.a1f798"],["e4d9c749.77e528"],["61531e4.e3881e"]],"outputLabels":["Dimmers","Switches",""]},{"id":"2abdc0d7.a1f798","type":"function","z":"65c335e0.d71ce4","name":"Dimmers initiate","func":"const inValues = msg.payload.values;\nconst Name = msg.payload.nodeName;\nconst zwnode = msg.payload.nodeId;\n\nlet endpoint;\nlet Brightness;\nlet zwClass = 38;\nlet onLevel;\nlet onLevelClass;\nlet minBright = 1;\n\ninValues.forEach(value => {\n    if (value.valueId.commandClass === zwClass && value.valueId.property == \"currentValue\") {\n        endpoint = value.valueId.endpoint;\n        Brightness = value.currentValue;\n    }\n    if (value.valueId.commandClass == 112) {\n        if (value.valueId.property == 18 || value.valueId.property == 32) {\n            onLevelClass = value.valueId.property;\n        }\n    }\n    if (value.valueId.commandClass == 112) {\n        if (value.valueId.property == 10 && value.valueId.propertyName == \"Minimum Brightness\") {\n            minBright = value.currentValue;\n        }\n    }\n});\n\nlet Dimmer = Name + endpoint;\n\nconst dimmerState = {\n    \"On\": Boolean(Brightness),\n    \"Brightness\": Brightness,\n    \"zwnode\": zwnode,\n    \"endpoint\": endpoint,\n    \"class\": zwClass,\n    \"onLevelClass\": onLevelClass,\n    \"minBright\" : minBright\n}\n\nglobal.set(Dimmer, dimmerState)\n\nconst newMsg = {\n    payload: {\n        \"On\": dimmerState.On\n    },\n    topic: Dimmer\n};\n\nif (dimmerState.On) {\n    newMsg.payload.Brightness = dimmerState.Brightness;\n}\n\nreturn newMsg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":380,"y":520,"wires":[["8dfc0fe6.4330c8"]]},{"id":"e4d9c749.77e528","type":"function","z":"65c335e0.d71ce4","name":"Switches initiate","func":"const inValues = msg.payload.values;\nconst Name = msg.payload.nodeName;\nconst zwnode = msg.payload.nodeId;\n\nlet endpoint;\nlet On;\nlet zwClass = 37;\n\ninValues.forEach(value => {\n    if (value.valueId.commandClass === zwClass && value.valueId.property == \"currentValue\") {\n        endpoint = value.valueId.endpoint;\n        On = value.currentValue;\n    }\n});\n\nlet Switch = Name + endpoint;\n\nconst switchState = {\n    \"On\": On,\n    \"zwnode\": zwnode,\n    \"endpoint\": endpoint,\n    \"class\": zwClass,\n}\n\nglobal.set(Switch, switchState)\n\nconst newMsg = {\n    payload: {\n        \"On\": switchState.On\n    },\n    topic: Switch\n};\n\nreturn newMsg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":380,"y":560,"wires":[["846a1f9.6d6cee"]]},{"id":"61531e4.e3881e","type":"function","z":"65c335e0.d71ce4","name":"Sensors initiate","func":"const inValues = msg.payload.values;\nconst Name = msg.payload.nodeName;\n\nlet outputs = [];\n\ninValues.forEach(value => {\n    if (value.valueId.commandClass === 128 && value.valueId.property == \"isLow\") {\n        outputs.push({\n            \"topic\": Name,\n            \"payload\": {\n                StatusLowBattery: + value.currentValue,\n                ChargingState: 2\n            }\n        })\n    }\n    if (value.valueId.commandClass === 128 && value.valueId.property == \"level\") {\n        outputs.push({\n            \"topic\": Name,\n            \"payload\": {\n                BatteryLevel: value.currentValue,\n                ChargingState: 2\n            }\n        })\n    }\n    if (value.valueId.commandClass === 113 && value.valueId.propertyKey == \"Door state\") {\n        let doorState = 23-value.currentValue;\n        if (isNaN(doorState)) {\n            doorState = 0;\n        }\n        outputs.push({\n            \"topic\": Name,\n            \"payload\": {\n                ContactSensorState: doorState\n            }\n        })\n    }\n    if (value.valueId.commandClass === 113 && value.valueId.propertyKey == \"Motion sensor status\") {\n        outputs.push({\n            \"topic\": Name,\n            \"payload\": {\n                MotionDetected: Boolean(value.currentValue)\n            }\n        })\n    }\n    if (value.valueId.commandClass === 113 && value.valueId.propertyKey == \"Sensor status\") {\n        if (value.valueId.property == \"CO Alarm\") {\n            outputs.push({\n                \"topic\": Name,\n                \"payload\": {\n                    CarbonMonoxideDetected: value.currentValue ? 1 : 0\n                }\n            })\n        }\n        if (value.valueId.property == \"Smoke Alarm\") {\n            outputs.push({\n                \"topic\": Name,\n                \"payload\": {\n                    SmokeDetected: value.currentValue ? 1 : 0\n                }\n            })\n        }\n        \n    }\n    if (value.valueId.commandClass === 49 && value.valueId.property == \"Air temperature\") {\n        outputs.push({\n            \"topic\": Name,\n            \"payload\": {\n                CurrentTemperature: value.currentValue\n            }\n        })\n    }\n    if (value.valueId.commandClass === 49 && value.valueId.property == \"Humidity\") {\n        outputs.push({\n            \"topic\": Name,\n            \"payload\": {\n                CurrentRelativeHumidity: value.currentValue\n            }\n        })\n    }\n    if (value.valueId.commandClass === 49 && value.valueId.property == \"Illuminance\") {\n        outputs.push({\n            \"topic\": Name,\n            \"payload\": {\n                CurrentAmbientLightLevel: value.currentValue\n            }\n        })\n    }\n});\n\nreturn [outputs];","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":300,"y":700,"wires":[["989e455.c5098b8"]]},{"id":"4e7ad117.fbe7d8","type":"link out","z":"65c335e0.d71ce4","name":"Door","links":["23ec89f1.7b13c6","13d54137.1b977f"],"x":670,"y":600,"wires":[],"l":true},{"id":"38487add.6c4bee","type":"link out","z":"65c335e0.d71ce4","name":"Motion","links":["d2b64b32.7ef778","a8f21129.18733","ef79915b.09d888","77765b56.4c0094"],"x":680,"y":560,"wires":[],"l":true},{"id":"de772036.9de7e","type":"link out","z":"65c335e0.d71ce4","name":"Temp","links":["c6d9055e.19a218"],"x":670,"y":640,"wires":[],"l":true},{"id":"3b937160.eb745e","type":"link out","z":"65c335e0.d71ce4","name":"Humidity","links":["63f3a0a6.d13c68"],"x":680,"y":680,"wires":[],"l":true},{"id":"79d7f8d8.ca1bb","type":"link out","z":"65c335e0.d71ce4","name":"Light","links":["7205045e.a74474"],"x":670,"y":720,"wires":[],"l":true},{"id":"87bd940b.04aea8","type":"link out","z":"65c335e0.d71ce4","name":"Battery","links":["aea2b5d.62b99c8"],"x":680,"y":760,"wires":[],"l":true},{"id":"8960042b.842a78","type":"link out","z":"65c335e0.d71ce4","name":"Smoke","links":["1ecf2ab7.ae934d"],"x":680,"y":800,"wires":[],"l":true},{"id":"5ee9c17b.513178","type":"link out","z":"65c335e0.d71ce4","name":"CO","links":["517361b6.27cad8"],"x":670,"y":840,"wires":[],"l":true},{"id":"989e455.c5098b8","type":"switch","z":"65c335e0.d71ce4","name":"","property":"payload","propertyType":"msg","rules":[{"t":"hask","v":"MotionDetected","vt":"str"},{"t":"hask","v":"ContactSensorState","vt":"str"},{"t":"hask","v":"CurrentTemperature","vt":"str"},{"t":"hask","v":"CurrentRelativeHumidity","vt":"str"},{"t":"hask","v":"CurrentAmbientLightLevel","vt":"str"},{"t":"hask","v":"ChargingState","vt":"str"},{"t":"hask","v":"SmokeDetected","vt":"str"},{"t":"hask","v":"CarbonMonoxideDetected","vt":"str"}],"checkall":"true","repair":false,"outputs":8,"x":470,"y":700,"wires":[["38487add.6c4bee"],["4e7ad117.fbe7d8"],["de772036.9de7e"],["3b937160.eb745e"],["79d7f8d8.ca1bb"],["87bd940b.04aea8"],["8960042b.842a78"],["5ee9c17b.513178"]]},{"id":"8dfc0fe6.4330c8","type":"link out","z":"65c335e0.d71ce4","name":"Dimmers initiate","links":["60ac2727.8fec2"],"x":535,"y":520,"wires":[]},{"id":"846a1f9.6d6cee","type":"link out","z":"65c335e0.d71ce4","name":"Switches initiate","links":["1118b4a3.f68dcb","c9773d67.7d53f"],"x":535,"y":560,"wires":[]},{"id":"f88b1f2b.ed12d","type":"switch","z":"65c335e0.d71ce4","name":"Ready","property":"status.text","propertyType":"msg","rules":[{"t":"eq","v":"All Nodes Ready!","vt":"str"}],"checkall":"true","repair":false,"outputs":1,"x":290,"y":400,"wires":[["a027a9ef.96ccb"]]},{"id":"a027a9ef.96ccb","type":"change","z":"65c335e0.d71ce4","name":"Get DB","rules":[{"t":"delete","p":"payload","pt":"msg"},{"t":"set","p":"payload","pt":"msg","to":"{\"class\":\"Driver\",\"operation\":\"GetValueDB\",\"params\":[]}","tot":"json"}],"action":"","property":"","from":"","to":"","reg":false,"x":450,"y":400,"wires":[["a0bc4ead.f6d5f8"]]},{"id":"5935956a.74874c","type":"link in","z":"65c335e0.d71ce4","name":"Value DB","links":["7902b802.cafbd8"],"x":95,"y":460,"wires":[["28dbecc1.f11fac"]]},{"id":"a0bc4ead.f6d5f8","type":"link out","z":"65c335e0.d71ce4","name":"Z-wave","links":["5ce2ebb.4955694"],"x":575,"y":400,"wires":[]},{"id":"93398c9d.fe96c","type":"inject","z":"65c335e0.d71ce4","name":"Get DB","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":270,"y":340,"wires":[["a027a9ef.96ccb"]]}]

[marcus-j-davies]

Haha!

Very nice! - Link nodes are Awesome!

Have been rocking the group ability!