Setup with scenes

[bartbutenaers]

Hi Simon,

Thanks for developing this contribution !

I want to setup a system with scenes (eg. 'at home', 'sleeping', 'to work').:

• When scene 'to work' is active, all sensors at home should be read.
• When scene 'sleeping' is active, some of the sensors upstairs shouldn't be read.
• ...

Some (common) flows should be active for multiple scenes, but other (dedicated) flows should be active only for a single scene.

There is also another contribution node-red-contrib-scenes that offers such functionality. But that node requires Node-Red to be started with their own executable, which is currently not what I want.

So the flows are all available, but only some of them should be active (based on the current scene): when the current scene changes I need to pass an array of flow names to one of your nodes to activate them (while all other flows should be deactivated).

Do you think such a setup could easily be implemented by using your contribution?

Or does this require a new extra node to be developed around following API function?

Thanks in advance !! Bart Butenaers

[btsimonh]

Hi Bart, first, thankyou for your interest! :) - this is the first feedback I've had, although the nodeset has been downloaded lots.... flowman allows you to add and delete flows; so when you want to load a scene, you could load a json containing the flows required, add the ones for that scene, and keep the returned IDs. Then when you want to change the scene, delete each of the flows which you stored the IDs for by id, and add the flows for the new scene. Another (maybe more robust) approach may be to prefix the label of all 'scene' flows with something you know (e.g. 'scene-'); then when it comes to deleting, you can just 'getflows' and iterate over all the present flows to delete ALL flows which are disposable before re-adding the ones you want from now on. (I'm thinking that when you 'addflow', the flow get's saved as part of your current flows; so if you have a machine crash/powerout, you may not know what flows it will start up on boot; but you could at this point just kill all 'scene-' flows and add the ones you know you should be running).

An alternative approach would be to work out how the enable flag is stored against a flow. The API you highlight activates a COMPLETE set of flows (and is the last thing that the red addflow calls, after adding a flow to the set...). I've not looked at the enabled flag, as it came along after I first wrote the node, but it seems a good addition to flowman would be an 'enableflow' node which took the flowid (or optionally a flow label?) plus an enable flag; this would suit you very well, because you could then have all your flows loaded (and easily editable), and just enable/disable as required.....

OK, I just had a look at the 0.17.5 NR code. Seems addflow/updateflow do not allow for the disabled flag to be passed in (they both create a brand new 'tab' node without these parameters). But there did seem to be plans for functions enableflow/disableflow, but they are currently set to null. A programmatic implementation without the features present in the API would be quite invasive (i.e. it would be far more related to red internals even than the current addflow/delflow implementation), and so probably not a great idea.

I'd suggest you go with the above addflow/delflow approach; I'll have a word with Nick and see if he wants contributions to enhance the programmatic API; I'm not sure if they have concrete plans around it for the moment.

best regards, simon

[bartbutenaers]

Hey Simon,

Thanks for your quick reply !! So I have 4 options to go (please correct me if I got it wrong!):

Option 1 - Add flow names and delete flow id's

I think that I would end up with a rather complex flow if I have to store/pass all the flow ID's myself. In my humble opinion this is internal kitchen of the Flowman nodes, and I would prefer to work with flow names only...

Option 2 - Add flow names and delete flow names

So I would have some kind of function node that handle's all the logic of adding and removing flows. Will create a test flow as soon as possible to test this.

Option 3 - Loading a complete set of flows

This is exactly what I'm aiming at, by using a simple flow of only 3 nodes:

• Dashboard node where user can change the current scene (e.g. a dropdown)
• Node to load the all flows of the selected scene
• Flowman node to set this complete set of flows The last node is the one I'm missing . Was really hoping that you were able to supply such a node ;-)

  Option 4 (??) - Enabling or disabling flows

  I think you mean this is currently not available yet?

Greetings from a rainy Belgium, Bart

[btsimonh]

I made an example of how I would see it working;
https://flows.nodered.org/flow/7a5147b61835690f470247d0f514d9b8 did find an issue too.... if you addflow or delflow in a loop, I think NR does not get the time for it's async saving. at first I thought it was overwriting stuff in my msgs and just doing the same flows over again; hence the strange stuff in the iteration functions.

So, basically.

1. create flows called template-ascenename. you can test and then disable these.
2. save them using the save feature (third chain) (set a directory in the fn node. left it at c:\datanobackup\scenes).
3. Add them using the 2nd chain (names specified in inject as json array).
4. delete all starting with 'scene-' with the first chain (ready to re-add a different set!).

just checked your github repos; you are playing with many of the same bits of NR as me, and I don't need to give detailed instructions :).

[btsimonh]

note the new issue about the addflow/delflow issue. (https://github.com/btsimonh/node-red-contrib-flowman/issues/2) In your flow design, also watch out for running delflow and addflow at close time proximity (I'm kind of assuming you will delflows, then addflows; so just put a short delay between them.

[bartbutenaers]

Hi Simon, thanks for all your effort.

I will try to experiment with all your tips this weekend, then I will give you feedback about my attempts.

Kind regards, Bart

[bartbutenaers]

Hello Simon,

I had some issues while installing your node. Seems I needed to add some dependencies to the package.json file, since there aren't any currently:

For example I got following error:

I created a fork of your project with following changes in the package.json file:

    "dependencies": {
        "fs-extra": "latest",
        "node-red": "latest"
    },

I installed my fork with this updated package.json file: npm install https://github.com/bartbutenaers/node-red-contrib-flowman.git

And now Node-Red starts without errors, and all your nodes are recognized in my flow editor. Especially the node-red dependency - necessary for the require('node-red') statement - seems to be a bit silly ...

However I don't understand that your other users didn't had the same installation errors ??? Any ideas?

Bart

[btsimonh]

hmm, fs-extra may be a new dependency; but i think it's already in node-red, so should not be required. However; I never use nodes in local/, I always have added them in the main node folders (i.e. they end up alongside node-red). We may find that if installed in local/, you now have two copies of node-red, and so the nodes may not actually work! Have a look and see exactly where it ended up, and where the deps got added (if at all). If you have ended up with a duplicate of NR, we may need to stipulate in install method. npm folder locations for deps are a black art to me... I'm guessing I don't see the issue because I install by putting it in my app package.json (I wrap NR).

Strange to not have had any feedback from others; maybe it's a case of people just downloading and abandoning immediately if the module does not work straight away.

s

[bartbutenaers]

Hi Simon,

Indeed the fs-extra is already a dependency for Node-Red:

I'm not an NPM expert, but I thought this is not relevant. Each node has it's own dependencies. So Node-Red can work with version X of fs-extra, while you are working with version Y. To accomplish this, every node has it's own node_modules subdirectory (containing the dependency nodes). E.g. on my Raspberry Pi, I get lots of nested node_modules directories:

/home/pi/.node-red/node_modules/fs-extra
/home/pi/.node-red/node_modules/node-red-contrib-flowman/**node_modules/fs-extra
/home/pi/.node-red/node_modules/node-red-contrib-flowman/node_modules/node-red
...

It think it is a bit stupid that you need to install Node-Red again for your contribution. Don't know if that is really necessary, because Node-Red will always be available in the super directory. Perhaps this can be solved by something like require("../../node-red") or require("../../node-red") instead of require("node-red"), to load the functionality from a super directory. I'm not at my computer currently, so I cannot try it now...

Still weird that no one reported this yet. Among all those users, there should be at least a single annoying person like me ;-)

[bartbutenaers]

Hi Simon,

I already failed during my first test of your flows, i.e. the one you provided me to save flows. While debugging, it seems that there are no flows available:

I assume that this is because I added node-red as a dependency, so it creates a new Node-Red instance (with a new empty json flow file)???

Secondly I tried to use the existing RED instance that is being passed:

But that RED instance doesn't contain the full API (as explained here): the RED instance e.g. has no getFlows() function. Probably that is why you are using the RED2 instance ...

And at last I tried to use RED2 = require('../../node-red'), in a desperate attempt to get the original RED instance. But that also fails:

I suppose reason is that I installed my Node-Red globally (to make sure it is accessible via system path):

sudo npm install -g --unsafe-perm node-red

As a result the node-red executable is in the /usr/bin directory (instead of in a root directory of your contribution), so the require statement cannot find it ...

I'll hate to admit it, but I have run out of creativity and knowledge ;-(

Kind regards, Bart

[btsimonh]

looks like you may have to try installing the node globally; I wonder if node-red will find it then; I got the go-ahead to look at the api from the team, so may also suggest that the API we need is made available to nodes... but this won't happen quickly as I'm up to my eyes in stuff which contributes to my bank balance rather than my ego :). I'll start another new issue specifically about installation location....

[bartbutenaers]

Simon,

don't worry about it. Your bank balance is more important. If a find a solution myself, I will keep you informed. Thanks for all your spare time you have spend!

Greetings, Bart

Op 14 aug. 2017 09:33 schreef "btsimonh" notifications@github.com:

looks like you may have to try installing the node globally; I wonder if node-red will find it then; I got the go-ahead to look at the api from the team, so may also suggest that the API we need is made available to nodes... but this won't happen quickly as I'm up to my eyes in stuff which contributes to my bank balance rather than my ego :).

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/btsimonh/node-red-contrib-flowman/issues/1#issuecomment-322119349, or mute the thread https://github.com/notifications/unsubscribe-auth/ANkLFcFVzwp6XfdEECsf7bdAsddzQrCCks5sX_hhgaJpZM4O1Gm4 .

[btsimonh]

Bart - could you try the following require lines in your configuration?

var fs = require.main.require("fs-extra");
var fspath = require("path");
var util = require('util');
var RED2 = require.main.require('node-red');

I've put my node-red-contrib-flowman inside local/node_modules, and it still worked with the original 'requires', but it also still works with the above. I wonder if this may solve your global node-red install?

I do not have a global install of node-red....

I've just pushed 0.0.8, which includes these require mods, and also serialises actions against the flow file so there will no longer be a need for crass delays or rate limits. I have not push it to NPM yet; will await to see if you can do some quick tests.

did one final update on top of 0.0.8 - to handle an error case a little better.

[bartbutenaers]

Hi Simon,

I have uninstalled the 0.0.7 version and installed your 0.0.8 directly from Github:

And indeed it seems to be working fine:

• No complains anymore in the startup log about missing Node-Red module
• In the debugger I see the RED2 variable is referring to my Node-Red instance. E.g. all my nodes are found:

The only problem I currently have is that my holiday ends tonight, so tomorrow I'll need to work at my bank balance ;-) I will try to experiment with the scene setup in the next weeks. Will keep you up-to-date.

Thanks a lot for all your support until now!!! Would not have figured out a solution with the require.main trick by myself in such a short notice.

Greetings from Belgium, Bart

[bartbutenaers]

Hello Simon,

Me again, a little later as expected ...

Have been playing with your flow and it works fine! So thanks again for publishing it! Did learn a lot by reading and debugging your code. Based on this new knowledge I started digging a little bit further and I experimented by adding a new node to your code: a SetFlows node to activate or deactivate flows. Perhaps ActivateFlows should be a better name...

Not fully completed yet, but I would appreciate if you would review it and let me know what you think about it. Because you are the flow manager in person ;-)

flows.html update

<script type="text/javascript">
    RED.nodes.registerType('setflows',{
        category: "flows",
        icon: "file.png",
        align: "right",
        color: "#AAEEAA",
        inputs: 1,
        outputs: 1,
        defaults: {
            name: { value: "" },
        },
        label: function() {
            return this.name || "setflows"
        }
    });
</script>

<script type="text/x-red" data-template-name="setflows">
    <div class="form-row">
        <label for="node-input-name"><i class="icon-tag"></i> Name</label>
        <input type="text" id="node-input-name" placeholder="Name">
    </div>
</script>

<script type="text/x-red" data-help-name="setflows">
   <p><b>sets the specified flow(s)</b></p>
   <p>input: <code>msg.id</code> containing a single flow id or an array of flow ids</p>
   <p>Output: <br>
   The input <code>msg</code> is passed to output with either <code>msg.id</code> and <code>msg.err</code> added/changed<br/></p>
   <p>Errors will be notified in <code>msg.err</code>, with <code>msg.payload = undefined</code></p>
</script>

flows.js update

     // set the flows from msg.id
    function setflows(n) {
        RED.nodes.createNode(this, n);
        this.name = n.name || '';
        var node = this;

        node.on('input', function (msg) {
            var flows = [];
            var flow = "";
            var node = {};

            if (typeof msg.id === 'string'){
                flows.push(JSON.parse(msg.id));
            } 

            if (!Array.isArray(msg.id)){
                node.error("did not setFlows - no flows found in .id");
                msg.err = "did not setFlows - no flows found in .id";
                return msg;
            }
            else {
                flows = msg.id;
            }

            try{
                for (var k = 0; k < flows.length; k++) {
                    flow = flows[k];
                    if (!getflow(flow)) {
                        node.error("did not setFlows - flow " + flow + " does not exist");
                        msg.err = "did not setFlows - flow " + flow + " does not exist";
                        return msg;                        
                    }
                }

                // read all flow ids from whole of node-red
                var allnodes = RED2.nodes.getFlows().flows;

                // Enable all flows specified in msg.id (and disable all other flows)
                for (var n = 0; n < allnodes.length; n++){
                    node = allnodes[n];
                    if (node.type === 'tab') {
                        node.disabled = !flows.includes(node.id);
                    }
                }                
            } catch (e) {
                delete msg.payload;
                msg.err = 'could not set flows ' + util.inspect(e);
                return msg;
            }

            try {                 
                // Set the flow nodes (with updated 'disable' fields)
                RED2.nodes.setFlows(allnodes);
            } catch (e) {
                delete msg.payload;
                msg.err = 'could not set flows ' + util.inspect(e);
            }

            node.send(msg);
        });
    }
    RED.nodes.registerType("setflows", setflows);
    //console.log("registered setflows");   
};

Explanation

In the msg.id field an array of flow (tab) id's is being specified. All flows that are being specified will be enabled, all other flows will be disabled.

Some test flow:

[{"id":"fa97f61e.568b58","type":"setflows","z":"6beebf75.ed0b5","name":"","x":1260,"y":340,"wires":[[]]},{"id":"c7893e19.4a52","type":"inject","z":"6beebf75.ed0b5","name":"","topic":"","payload":"[\"47b91ceb.38a754\",\"d7a683c8.03b5e\",\"985aac0b.bcc38\",\"d0d4f76e.209b68\",\"6beebf75.ed0b5\",\"4900f0c0.a1ad6\",\"942c4e5e.b137a\",\"f46bc5ef.dcbab8\"]","payloadType":"json","repeat":"","crontab":"","once":false,"x":930,"y":340,"wires":[["19fac069.fbec5"]]},{"id":"19fac069.fbec5","type":"change","z":"6beebf75.ed0b5","name":"","rules":[{"t":"set","p":"id","pt":"msg","to":"payload","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":1090,"y":340,"wires":[["fa97f61e.568b58"]]}]

This way I can store for each scene an array of all flow ids that are relevant for that scene. When a scene is activated by the user, I simply send the array of flow ids to the setflow node (and only those flows are enabled, all others disabled)...

In this test I passed an array of all flow id's, except from flow id "c795f3e1.2fc81" which is being disabled by my node:

So the setflows node does a little bit the similar to pressing the flow's disabled button:

I assume this has something to do with your Node-Red issue, but not 100% clear to me what ...

Is this a good solution, a bad solution, do you have a better proposal? Very curious about your opinion !

Kind regards, Bart Butenaers

[bartbutenaers]

Simon (@btsimonh),

An extra question about your issue. You and Nick are only discussing addFlow/getFlow/updateFlow, while I'm using the setFlows. Is there any reason why the setFlows is not mentioned? Is there some disadvantage of setFlows perhaps, I'm not aware of?

[btsimonh]

Hi bart, will review next week as i'm in amsterdam at our major european show (with my flemmish colleagues :) ). Note the issue in NR i've raised about updating the programatic interface for enable/disable also... have basic agreement from nick that i can work on this, just a matter of time... Br, Simon

On Sep 14, 2017 10:08 PM, bartbutenaers notifications@github.com wrote:

Hello Simon,

Me again, a little later as expected ...

Have been playing with your flow and it works fine! So thanks again for publishing it! Did learn a lot by reading and debugging your code. Based on this new knowledge I started digging a little bit further and I experimented by adding a new node to your code: a SetFlows node to activate or deactivate flows. Perhaps ActivateFlows should be a better name...

Not fully completed yet, but I would appreciate if you would review it and let me know what you think about it. Because you are the flow manager in person ;-)

flows.html update

flows.js update

 // set the flows from msg.id
function setflows(n) {
    RED.nodes.createNode(this, n);
    this.name = n.name || '';
    var node = this;

    node.on('input', function (msg) {
        var flows = [];
        var flow = "";
        var node = {};

        if (typeof msg.id === 'string'){
            flows.push(JSON.parse(msg.id));
        }

        if (!Array.isArray(msg.id)){
            node.error("did not setFlows - no flows found in .id");
            msg.err = "did not setFlows - no flows found in .id";
            return msg;
        }
        else {
            flows = msg.id;
        }

        try{
            for (var k = 0; k < flows.length; k++) {
                flow = flows[k];
                if (!getflow(flow)) {
                    node.error("did not setFlows - flow " + flow + " does not exist");
                    msg.err = "did not setFlows - flow " + flow + " does not exist";
                    return msg;
                }
            }

            // read all flow ids from whole of node-red
            var allnodes = RED2.nodes.getFlows().flows;

            // Enable all flows specified in msg.id (and disable all other flows)
            for (var n = 0; n < allnodes.length; n++){
                node = allnodes[n];
                if (node.type === 'tab') {
                    node.disabled = !flows.includes(node.id);
                }
            }
        } catch (e) {
            delete msg.payload;
            msg.err = 'could not set flows ' + util.inspect(e);
            return msg;
        }

        try {
            // Set the flow nodes (with updated 'disable' fields)
            RED2.nodes.setFlows(allnodes);
        } catch (e) {
            delete msg.payload;
            msg.err = 'could not set flows ' + util.inspect(e);
        }

        node.send(msg);
    });
}
RED.nodes.registerType("setflows", setflows);
//console.log("registered setflows");

};

Explanation

In the msg.id field an array of flow (tab) id's is being specified. All flows that are being specified will be enabled, all other flows will be disabled.

Some test flow: [image]https://user-images.githubusercontent.com/14224149/30452693-da00638a-9996-11e7-9a53-30bb7c75989e.png

[{"id":"fa97f61e.568b58","type":"setflows","z":"6beebf75.ed0b5","name":"","x":1260,"y":340,"wires":[[]]},{"id":"c7893e19.4a52","type":"inject","z":"6beebf75.ed0b5","name":"","topic":"","payload":"[\"47b91ceb.38a754\",\"d7a683c8.03b5e\",\"985aac0b.bcc38\",\"d0d4f76e.209b68\",\"6beebf75.ed0b5\",\"4900f0c0.a1ad6\",\"942c4e5e.b137a\",\"f46bc5ef.dcbab8\"]","payloadType":"json","repeat":"","crontab":"","once":false,"x":930,"y":340,"wires":[["19fac069.fbec5"]]},{"id":"19fac069.fbec5","type":"change","z":"6beebf75.ed0b5","name":"","rules":[{"t":"set","p":"id","pt":"msg","to":"payload","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":1090,"y":340,"wires":[["fa97f61e.568b58"]]}]

This way I can store for each scene an array of all flow ids that are relevant for that scene. When a scene is activated by the user, I simply send the array of flow ids to the setflow node (and only those flows are enabled, all others disabled)...

In this test I passed an array of all flow id's, except from flow id "c795f3e1.2fc81" which is being disabled by my node:

[image]https://user-images.githubusercontent.com/14224149/30453011-b8cbbfe2-9997-11e7-8f12-ae9969af1453.png

So the setflows node does a little bit the similar to pressing the flow's disabled button: [image]https://user-images.githubusercontent.com/14224149/30453152-31388da2-9998-11e7-90bb-b82914acccd1.png

I assume this has something to do with your Node-Red issuehttps://github.com/node-red/node-red/issues/1372, but not 100% clear to me what ...

Is this a good solution, a bad solution, do you have a better proposal? Very curious about your opinion !

Kind regards, Bart Butenaers

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/btsimonh/node-red-contrib-flowman/issues/1#issuecomment-329594435, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AC_fNE6Do8JVRKKO7TWNq4LBIVU9hNNJks5siYergaJpZM4O1Gm4.

[bartbutenaers]

Goedemiddag Simon, veel plezier op de Europese show ;-) Groetjes, Bart

[btsimonh]

I'll get them to translate when they get back from partying! S

On Sep 15, 2017 12:46 PM, bartbutenaers notifications@github.com wrote:

Goedemiddag Simon, veel plezier op de Europese show ;-) Groetjes, Bart

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/btsimonh/node-red-contrib-flowman/issues/1#issuecomment-329748083, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AC_fNJeWtxAutIloFRr9fF7BSqm0m6PVks5silV2gaJpZM4O1Gm4.

[arman2019b]

Hello Bart/Simon, I just randomly come across this issue with hope if I can find my answer here. actually, I'm very basic with node-red and trying to switch enable/disable a flow tab. I can obtain the flow id by getflows node, but I can't figure out how to disable it?

please let me know your comment.

[btsimonh]

hi arman2019b, As I found out yesterday when I inadvertently updated the node-red and flowman to latest in my production build, flow.disabled is now supported in addflow and getflow. Originally it was not supported by NodeRed, but this commit corrects that. Not sure what minimum NR version you would need (I'm still on an old 0.17, modified). This node set does not help in changing it on a current flow, but you could easily delflow and then addflow with the modified value. Just be aware that subflows will be squashed into the main flow and reproduced, as will global config nodes, and that the variable 'disabled' will only be present in the flow if it is disabled... so getflow a disabled one to see where to add it :). br, Simon

[arman2019b]

hi, do you have by chance any sample file that i can use?