Sparkplug C addition

[Llorx]

Able to send a device as a Template instance. Maybe you have an "Inverter" Template and you have a device INV001 which is an instance of the "Inverter" Template. To send this data inside a device in Sparkplug, you have to send a metric with a name, which is weird as you may have a device named INV001 with a metric with an invented name (like inverter, or instance or whatever) that defines the Template instance.

In the industrial world is difficult to make someone understand that a device that has instanced 100 times in his model is not going to be sent as a Template and Template instances, but separated devices with no link to a predefined Template, unless you invent a metric to drop the instance.

I suggest that metrics could contain more metrics (like groups?) so a device could be a metric itself, which could countain more metrics, or have a value itself (which may be a Template instance).

This may be silly, but is usual and people is not going to force their model to fit Sparkplug, but want protocols that fit their model instead. Even in a recent HiveMQ webinar talking about UNS they had this model: Notice Bottler, which is the device, which should be a "Bottler" instance, but with Sparkplug is impossible. We wanted to show Sparkplug benefits in that webinar (We are N3uron) but it was impossible to send Bottler instances without workarounds, so we had to show a model without templates.

[matthewGEA]

Does this existing issue cover it? https://github.com/eclipse-sparkplug/sparkplug/issues/130

[Llorx]

Does this existing issue cover it? #130

No. Is a different thing.

[matthewGEA]

Given your are speaking from the perspective of N3euron, as a Sparkplug client, your connection is properly understood as an Edge of Network node. The device concept was originally intended to represent a legacy device that is connected to this node. Therefore, the device itself is represented (through the device name) and all of the data that the device owns is included underneath.

Given that N3uron is likely modeling data on behalf of devices, could you comment on the need to use the Device mechanism? It seems that you are trying to have modeled metrics appear under a Device, rather than having your device model template be instantiated under the Node namespace.

Thanks for the clarification.

[Llorx]

Today we had a webinar with HiveMQ, Opto22 (using Ignition edge) and Sepasoft to show the benefits of having a Unified Namespace. For this, we all pushed our data via Sparkplug to a MQTT broker for the application to consume, and having a namespace defined for everyone to follow, this problem arised.

The typical scenario is to have nodes and devices under those nodes. The usual thing for those devices is to have plain metrics, for example a bottle filler will have a metric with the amount of bottles per minute filled.

With this scenario, the group is the factory location, the node is the edge node that collects the data and the device is the bottle filler. Is usual to have multiple bottle fillers, so a UDT (a.k.a. Template) is used to design it. When "converting" the model in Ignition edge to Sparkplug, you set the fillers as devices, so this topic as per /group/node/device is created: /Madrid/Line01/Filler01

At this point is impossible to tell that Filler01 itself is an instance of a UDT. When you talk to whatever integrator about devices, the usual thing to think is about a piece of hardware used in multiple lines or zones, not a singleton, hence templates is the way to go in the model to define them (Ignition, N3uron or whatever data collector).

[matthewGEA]

Any reason your can't communicate the model (UDT) as a metric underneath the Node directly, using the Speaking Template mechanism?

It would still appear as Madrid/Line/Filler, but Filler would be an instance of the UDT.

I'm trying to zero-in on the specific need to publish via Node/Device/UDTinstanceName instead of Node/UDTinstanceName

[Llorx]

Because it adds noise to the unified namespace.

When you convert those metrics again to "slash paths" instead of /Madrid/Line01/Filler01/BottlesPerMinute you are going to end with /Madrid/Line01/Filler01/Filler01(again because is the metric name with the UDT instance)/BottlesPerMinute which is ugly and counterintuitive of what a device instance is.

If we could somehow tell that this device is a template instance and that its metrics are defined in the template, the unified namespace would be more clean instead of adding noise.

Now if you create a plain device with a tag inside, without being a UDT instance, you end up with /Madrid/Line01/Filler01/BottlesPerMinute, but if you decide to covert the device to a UDT because you bought another filler, you suddenly end with /Madrid/Line01/Filler01/Filler01/BottlesPerMinute. Feels weird behaviour.

[matthewGEA]

When you generate the NBIRTH packet in group=Madrid and node=Line01, are you including a template definition per Sparkplug spec section 6.4.14. Template...defining the template definition name = fillers (for example) and set is-definition=true?

Then are you defining a metric within the NBIRTH packet with name filler01 (for example) as a datatype "template" with template_ref = fillers?

This shouldn't add noise to the namespace...

[Llorx]

I see, a metric inside the node instead of a device.

That checks partially but is the user the one doing the "add device, point to instance X and why I don't receive a template instance for this device?" and you will need to explain him that he needs to add the device as a metric for the node instead.

If you send devices as node metrics, well, is not a node metric, but a device disghished as a node value, which should be values linked to the node status instead. Seems a workaround but checks when converting back to "slash paths" so thank you for the tip. By the way, as Sparkplug C is going to be redefined, thinking a way to fix this would be awesome.

I have in mind to add the templateRef property to devices also, so this will force those devices to follow the template metrics like any other metric that has a templateRef.

[matthewGEA]

In Sparkplug, there is no real definition of "Node" or "Device", except that the Node is the software actually publishing MQTT packets and a device is an optional, abstract concept.

I can imagine a piece of software that is a Node with no devices, and yet publishes DDATA as virtual devices because it wants to organize its data in that way.

Also, I can imagine a piece of software that is a Node with several devices, but these devices have no inherent capability for modeling its data, and so the node is organizing and modeling the device's data, becoming the custodian/owner of the data, and therefore publishes its modeled data as metrics under the node's namespace.

Regarding how the UDT will appear within a given software that is aggregating the data for visualization, it's up to the producer on how the UDT should appear...should it appear underneath the NODE or should it appear underneath the DEVICE?

Additionally, how Ignition MQTT Engine organizes the data is not defined by the sparkplug standard. For example, representing groupName/DDATA/NodeName/DeviceName containing metric of UDTname as groupName/NodeName/DeviceName/UDTname is a design choice that the MQTT Engine product has made. Other software instead could organize the same exact data being received as: groupName/Nodes/NodeName/Devices/DeviceName/UDTs/UDTName or UnmodeledMetrics/metricName , as a flat list where each metric gets metadata defining groupName, NodeName and DeviceName with a second folder of UDTtype/UDTname again embedding metadata that defines the UDT groupName, NodeName and DeviceName.

All this to say, Sparkplug doesn't define how a software consumes the data being sent across the wire and there is no strict definition on how to use "Devices". If you think it is messy for Ignition to show a device UDT instance that represents an entire device appear underneath the name of a Device, then I suggest not using "devices" and put your devices UDT directly within your Node namespace, or organize it however you want using slashes in the metric name.

What is certain, is the node is the one publishing the data. It's not against the standard for the node to include data from other sources within its own namespace.

If you are thinking of the user's experience when modeling data within software, maybe provide the user a choice whether to publish a given UDT instance within the NODE or NODE/DEVICE namespace. Devices can instantiate templates within their DBIRTH messages after the NBIRTH message publishes the definition. So the function is there, but it seems to go against your preference on how this is visualized within a tool like Ignition. I suspect the reason you can't define a whole DEVICE as a UDT is the same reason you can't define a whole NODE as a UDT...both a Node and a Device can publish multiple instances of multiple UDT types so they are simply used as "container names" for whatever data within them.

[RickBullotta]

Any reason your can't communicate the model (UDT) as a metric underneath the Node directly, using the Speaking Template mechanism?

It would still appear as Madrid/Line/Filler, but Filler would be an instance of the UDT.

I'm trying to zero-in on the specific need to publish via Node/Device/UDTinstanceName instead of Node/UDTinstanceName

This is precisely why durable metadata is essential. Also, if templates/types get published somewhat randomly from anywhere, how will conflicts and inconsistencies be dealt with?