Is there a way to have the data in the 2.0 chart node survive restarts?

[houser42]

Description

The V1 chart has a tested and working subflow that saves the data and reloads the data after a restart. This is pretty vital functionality and needed for V2 charts. Can someone have look at this when there is time please?

I tested the old working subflow with the V 2 chart and as expected it does not work. It would be great if this subflow could be updated to work with 2.0. or of course any other way to have the data survive reboots if possible? This is the subflow that worked with V1 hooked up to a V2 chart for tests:

[
    {
        "id": "3b6a1337b69e1574",
        "type": "subflow",
        "name": "vvb d2",
        "info": "",
        "category": "",
        "in": [
            {
                "x": 50,
                "y": 30,
                "wires": [
                    {
                        "id": "0803c4cc5eb6de01"
                    }
                ]
            }
        ],
        "out": [
            {
                "x": 820,
                "y": 160,
                "wires": [
                    {
                        "id": "1d445f73ea677ec2",
                        "port": 0
                    },
                    {
                        "id": "de6cdd9bd32b3eb1",
                        "port": 0
                    }
                ]
            }
        ],
        "env": [],
        "meta": {},
        "color": "#DDAA99"
    },
    {
        "id": "1d445f73ea677ec2",
        "type": "file",
        "z": "3b6a1337b69e1574",
        "name": "",
        "filename": "",
        "appendNewline": true,
        "createDir": true,
        "overwriteFile": "true",
        "x": 660,
        "y": 140,
        "wires": [
            []
        ]
    },
    {
        "id": "006debb5e922bb46",
        "type": "file in",
        "z": "3b6a1337b69e1574",
        "name": "",
        "filename": "",
        "format": "utf8",
        "sendError": true,
        "x": 500,
        "y": 180,
        "wires": [
            [
                "de6cdd9bd32b3eb1"
            ]
        ]
    },
    {
        "id": "e37d519cbbd162d3",
        "type": "json",
        "z": "3b6a1337b69e1574",
        "name": "",
        "x": 500,
        "y": 140,
        "wires": [
            [
                "1d445f73ea677ec2"
            ]
        ]
    },
    {
        "id": "de6cdd9bd32b3eb1",
        "type": "json",
        "z": "3b6a1337b69e1574",
        "name": "",
        "x": 660,
        "y": 180,
        "wires": [
            []
        ]
    },
    {
        "id": "0803c4cc5eb6de01",
        "type": "function",
        "z": "3b6a1337b69e1574",
        "name": "LoadSave",
        "func": "var strSafe=msg.topic;\nif(strSafe)\n{\n    strSafe=strSafe.replace(/[^a-z0-9]/gi, '_').toLowerCase();\n    var loaded=context.get(strSafe)||0;\n    msg.filename =\"/home/hb19/.node-red/flowsave/vvbd2/\"+strSafe+\".dat\";\n    //node.warn(loaded);\n    if(0===loaded)\n    {\n        //load chart!! \n        msg.payload=\"load\";\n        context.set(strSafe,1);\n        \n        //node.warn(\"Loading \" +strSafe);\n    }\n    else\n    {\n        //node.warn(\"Writing \" +strSafe);\n    }\n    return msg;    \n}\nreturn null;\n\n",
        "outputs": 1,
        "timeout": "",
        "noerr": 0,
        "initialize": "",
        "finalize": "",
        "libs": [],
        "x": 200,
        "y": 160,
        "wires": [
            [
                "abd9aa795d47fbc6"
            ]
        ]
    },
    {
        "id": "abd9aa795d47fbc6",
        "type": "switch",
        "z": "3b6a1337b69e1574",
        "name": "Load data",
        "property": "payload",
        "propertyType": "msg",
        "rules": [
            {
                "t": "neq",
                "v": "load",
                "vt": "str"
            },
            {
                "t": "eq",
                "v": "load",
                "vt": "str"
            }
        ],
        "checkall": "false",
        "repair": false,
        "outputs": 2,
        "x": 350,
        "y": 160,
        "wires": [
            [
                "e37d519cbbd162d3"
            ],
            [
                "006debb5e922bb46"
            ]
        ]
    },
    {
        "id": "3dc8c1a1d345ef1d",
        "type": "ui-chart",
        "z": "98b31619.daed3",
        "g": "120d335300e6531b",
        "group": "f83d15d3a238accd",
        "name": "vvb chart week",
        "label": "",
        "order": 4,
        "chartType": "line",
        "category": "topic",
        "categoryType": "msg",
        "xAxisLabel": "",
        "xAxisProperty": "",
        "xAxisPropertyType": "property",
        "xAxisType": "time",
        "xAxisFormat": "",
        "xAxisFormatType": "auto",
        "yAxisLabel": "",
        "yAxisProperty": "",
        "ymin": "15",
        "ymax": "65",
        "action": "append",
        "stackSeries": false,
        "pointShape": "line",
        "pointRadius": "2",
        "showLegend": false,
        "removeOlder": 1,
        "removeOlderUnit": "604800",
        "removeOlderPoints": "",
        "colors": [
            "#ff9300",
            "#00fdff",
            "#ff7f0e",
            "#2ca02c",
            "#a347e1",
            "#d62728",
            "#ff9896",
            "#9467bd",
            "#c5b0d5"
        ],
        "textColor": [
            "#ffffff"
        ],
        "textColorDefault": false,
        "gridColor": [
            "#7a7a7a"
        ],
        "gridColorDefault": false,
        "width": "30",
        "height": "10",
        "className": "",
        "x": 4420,
        "y": 540,
        "wires": [
            [
                "2e85ceab3abb96f6"
            ]
        ]
    },
    {
        "id": "2e85ceab3abb96f6",
        "type": "subflow:3b6a1337b69e1574",
        "z": "98b31619.daed3",
        "g": "120d335300e6531b",
        "name": "",
        "x": 4430,
        "y": 600,
        "wires": [
            [
                "3dc8c1a1d345ef1d"
            ]
        ]
    },
    {
        "id": "f83d15d3a238accd",
        "type": "ui-group",
        "name": "vvb",
        "page": "40ab10feb1ddf2bb",
        "width": "30",
        "height": "20",
        "order": 1,
        "showTitle": false,
        "className": "",
        "visible": "true",
        "disabled": "false"
    },
    {
        "id": "40ab10feb1ddf2bb",
        "type": "ui-page",
        "name": "vvb",
        "ui": "cd62f850b848415c",
        "path": "/vvb",
        "icon": "home",
        "layout": "grid",
        "theme": "404b2b83a6155d6c",
        "breakpoints": [
            {
                "name": "Default",
                "px": "0",
                "cols": "30"
            },
            {
                "name": "Tablet",
                "px": "1080",
                "cols": "30"
            },
            {
                "name": "Small Desktop",
                "px": "1080",
                "cols": "30"
            },
            {
                "name": "Desktop",
                "px": "1080",
                "cols": "30"
            }
        ],
        "order": 1,
        "className": "",
        "visible": true,
        "disabled": false
    },
    {
        "id": "cd62f850b848415c",
        "type": "ui-base",
        "name": "flax 24",
        "path": "/dashboard",
        "includeClientData": true,
        "acceptsClientConfig": [
            "ui-notification",
            "ui-control"
        ],
        "showPathInSidebar": false,
        "showPageTitle": false,
        "navigationStyle": "default",
        "titleBarStyle": "default"
    },
    {
        "id": "404b2b83a6155d6c",
        "type": "ui-theme",
        "name": "dark jtm",
        "colors": {
            "surface": "#000000",
            "primary": "#0094ce",
            "bgPage": "#000000",
            "groupBg": "#000000",
            "groupOutline": "#000000"
        },
        "sizes": {
            "pagePadding": "12px",
            "groupGap": "12px",
            "groupBorderRadius": "4px",
            "widgetGap": "12px",
            "density": "comfortable"
        }
    }
]

Have you provided an initial effort estimate for this issue?

I am no FlowFuse team member

[Tragl1968]

Is there a way to up-vote this basic request? "a way to have the data in the 2.0 chart node survive restarts"

[joepavitt]

Further investigation is required on our part on how we can support this. We can't support this natively in the module, as when node-RED restarts, all temporary data is cleared.

The flow shared utilises a file-in/file-out out, which we can investigate, and provide a sample flow for.

[colinl]

I think there will be ongoing requests for this feature. The D1 node accomplishes it by providing an output which can can be saved in file or context and directly re-injected on startup, which is easy for the user to implement and does the job perfectly.

[Tragl1968]

1) Retention of data points within v2.0 (possible to-do flowuse) In my opinion, a perfect basis would be if the ui-graph in version 2.0 would generate an output like in version 1.0. Namely all >displayed< data points (older no more displayed data points are obsolete) as an array, instead of just the last data point received. (The array must actually already be there internally, because how else does the graph display the data points for newly calling clients?)

2) Saving and injecting (to-do for someone else) There is already a simple external option for permanently saving and then injecting the output array, e.g. with PERSIST. Alternatively, you can adapt the flow mentioned in this thread.

[joepavitt]

Yes, I am considering the unified output approach, but we'd also then need to add support for injecting that data format too.

[colinl]

we'd also then need to add support for injecting that data format too.

Could it output data compatible with the existing config of the node, or would that be difficult to achieve?

[houser42]

Hello. Just some positive feedback and also a question that I would assume I will not get an answer to as it is a bit of a "how long is a string" type of question. But here it goes:

We have finished migrating 3 offgrid systems from Dashboard V1 to Dashboard V2 that are now only missing this feature. It has gone much better than expected. D2 seems very stable and robust so far with much better performance than D1. So very happy about that and kudos for great work and the migration tools and docs!

If at all possible. we would like to deploy these systems March-April next year, so is it realistic to hope for this feature by then by any chance? Just planning ahead, and the V1 systems can certainly last for a while more so not vital but would be nice as maintenance for V1 has become a tad cumbersome and stale.

To be clear: A way to have the data in the 2.0 chart node save the data and reload that data / survive restarts.