Feature request: alias instead of metric name

[nonujigu]

Inside the payload of the DDATA messages, all metric names appear in plain text. The metric names have an immense impact on the size of the resulting Sparkplug B payload. This really hurts when the edge device sends the data to the broker via 3G/4G/5G.

I played around with the encodePayload function within the sparkplug-payload package. It turns out it supports metric name aliasses:

The "traditional" way works as expected:

{
    "metrics": [
        {
            "name": "ambientAirTemperature",
            "value": 28.7,
            "type": "FLOAT",
            "timestamp": 1668357518266
        },
        {
            "name": "ambientAirPressure",
            "value": 33.6,
            "type": "FLOAT",
            "timestamp": 1668357518266
        }
    ],
    "seq": 1
}

But also the "alias" way works without errors:

{
    "metrics": [
        {
            "alias": 1,
            "value": 28.7,
            "type": "FLOAT",
            "timestamp": 1668357518266
        },
        {
            "alias": 2,
            "value": 33.6,
            "type": "FLOAT",
            "timestamp": 1668357518266
        }
    ],
    "seq": 1
}

The resulting buffer of the encodePayload function has a size of 77 bytes in the "traditional" way vs. 38 bytes in the "alias" way. I was able to decode both buffers with the corresponding decodePayload function.

As far as I understand the Sparkplug B specification, alias (without name) can be utilized in DDATA messages if both name and alias have been exposed in the DBIRTH message. Maybe it is not that useful to implement alias properties in the UI. However, it would be great to have this feature in dynamic metric definitions.

[thomassorensen2650]

I took a quick look at the code, and it should be pretty straightforward to implement a solution that would work for both dynamic and static metrics by modifying createMsg to lookup metric id, If the user has has enabled the feature.

I'll take a look at it when I have some free time.

[nonujigu]

Maybe it is worth considering that the alias of a metric is just calculated by the order in which the metric appears in the definition. A lookup could then be based on an array which would be very cheap in terms of cpu and ram usage. This could have a significant performance advantage if Node-RED is running on weaker hardware like a single core pi zero.

However, as far as I understand the sparkplug b specification, aliases are positive integers that must be unique for each metric, but not necessarily in an order starting from 0.

Another thing to consider is that aliases must be unique across the whole EoN, not just a single sparkplug device.

[nonujigu]

I found a solution by modifying the code that seems to works for me. I utilized a Map object for a lookup function and deleted the name property inside the metrics array elements if the lookup was successful. Every time a new definition is received, this new lookup Map is initialized by scanning the definition for an "alias" property.

So a definition would look like this:

msg = {
    "definition": {
        "metric_1": {
            "alias": 1,
            "dataType": "Float"
        },
        "metric_2": {
            "alias": 2,
            "dataType": "Float"
        }
    }
};

The payload is still expected to have the name attribute in every metrics element.

I utilized MQTT.fx 1.7.1 with the built in sparkplug decoder for debugging. This is the decoded DBIRTH message:

{
    "timestamp": 1668599875082,
    "metrics": [
        {
            "name": "metric_1",
            "alias": 1,
            "timestamp": 1668599875080,
            "dataType": "Float",
            "value": 1.0
        },
        {
            "name": "metric_2",
            "alias": 2,
            "timestamp": 1668599875080,
            "dataType": "Float",
            "value": 2.0
        }
    ],
    "seq": 1
}

This is the following decoded DDATA message:

{
    "timestamp": 1668599876083,
    "metrics": [
        {
            "name": "",
            "alias": 1,
            "timestamp": 1668599876081,
            "dataType": "Float",
            "value": 0.1470015
        },
        {
            "name": "",
            "alias": 2,
            "timestamp": 1668599876082,
            "dataType": "Float",
            "value": 2.7654445
        }
    ],
    "seq": 2
}

Somehow the MQTT.fx sparkplug decoder still outputs the name as an empty string. I don't know why this is the case. It could be that this is just the normal behavior of the MQTT.fx sparkplug decoder. I guess I will continue testing this with other sparkplug decoders.

[thomassorensen2650]

The feature has been fully implemented in this PR #13

I'll see if I can merge that in and release a new version within the next couple of days.

[nonujigu]

The feature has been fully implemented in this PR #13

I'll see if I can merge that in and release a new version within the next couple of days.

Thanks @thomassorensen2650 for this great feature update. I just tested it and I can confirm that it behaves as expected. I can see that you also support auto generated aliases for static metrics.

Please consider the following suggestions as "whining on a high level" as your library is by far the best implementation for Node-RED:

Allow custom aliases in dynamic definitions

As far as I can see, the aliases 1 and 2 are currently reserved for metrics for NBIRTH/NDATA messages. Aliases for DBIRTH/DDATA are starting at 3. It would be nice if the mqtt sparkplug device node would accept custom aliases in msg.definition:

msg = {
    "definition": {
        "metric_1": {
            "alias": 5,
            "dataType": "Float"
        },
        "metric_2": {
            "alias": 6,
            "dataType": "Float"
        }
    }
};

The advantage of this approach would be a better compatibility with older versions of metric definitions, e.g. if new metrics are added in a definition, the aliases of the old metrics would not necessarily have to change.

Retrieve static metric definitions

It might be useful to be able to get the static metric definition (including auto generated aliases) inside a flow, e.g. for storing them in flow context. This could be possibly achieved by utilizing a second output for the mqtt sparkplug device node that sends the definition after it has been initialized.

Allow incoming metrics to contain an alias property instead of a name property.

In case that aliases have been defined, it might be useful to support incoming metrics that already contain an alias instead of a name:

msg.payload = {
    timestamp: 1668674090878,
    metrics: [
        {
            timestamp: 1668674090878,
            alias: 1,
            value: 1.5249621827007886
        },
        {
            timestamp: 1668674090878,
            alias: 2,
            value: 2.147691124220938
        }
    ]
};

Automatically resolve aliases to names

Currently, the mqtt sparkplug in node outputs the metric name of DDATA messages as empty strings if the sparkplug payload contains aliases instead of names. It would be nice if the node could resolve those names automatically according to the latest corresponding DBIRTH message. The output of the alias is divided into high and low. This is probably because alias is a 64-bit integer within the sparkplug payload.

By the way: The mqtt sparkplug in node outputs the timestamp as a string, probably because timestamp is a 64-bit integer within the sparkplug payload and a coercion to a javascript number could lead in theory to a coercion issue. However, it would be nice if timestamp would be a javascript number.

[thomassorensen2650]

I'll keep the suggestions in mind for next version :).. for now I'll close the ticket as the functionality has been implemented 1.3.2.

[archerixx]

Hi, I am sorry for reopenning old Issue, but I am hoping either @nonujigu or @thomassorensen2650 could give answer to this.

What is the way to know what aliases were given to each topic inside Node-red ? What I am trying to do, is to send message towards Node-red from Ignition. I do receive alias with metric value, but I do not know to which topic does that alias corresponds. That way, I dont know what to activate on my logic side inside Node-red.

I figured out there is shallow copy when metrics are sent from SparkplugB client node, so currently I am extracting aliases that way. After shallow copy does its thing, "name" object is removed as @nonujigu said, so that same metrics can not be sent again to sent through SparkplugB. I made workaround, so when aliases are set inside shallow copy object, I then continue using deep copy so name can be preserved and alises are dealt only inside SparkplugB node.

I feel like I am missing something here, so I would like your opininos on this topic. Is there better way to connect topics and aliases on Node-red side, so I may connect DCMD data that comes from Ignition (including only value and alias) to actual topic that was initialy sent towards it. I can you give example of my flow if needed.

We can move this to new issue, I just wanted to try reach @nonujigu for this info as well.

Thank you