coussej / node-opcua-logger

An OPCUA Client for logging data to InfluxDB! 🔌 🏭
https://www.factry.io
MIT License
180 stars 64 forks source link

Polling multiple fields in one shot #58

Closed mirkocomparetti-synesis closed 3 years ago

mirkocomparetti-synesis commented 3 years ago

Hello, Is it possible (and how) to configure the tool to poll read two nodes and write them as separated fields in influxdb?

Thanks, Mirko

coussej commented 3 years ago

Hello,

no, each node goes to a different measurement in influxdb. However, when you use polling at the same frequency, they will have exactly the same timestamp zo you can easily query them together.

Regards, Jeroen

mirkocomparetti-synesis commented 3 years ago

Picture this use case: Imagine an industrial application where you have a process that is composed by 10 subsystems generating 20 signals each: having 200 measurement isn't a little too much and not optimized? To show all data from one subsystem you have to "look" into 10 tables instead of one.

Thanks

coussej commented 3 years ago

We have multiple usecases like you describe, with 1000s of signals. When you have a structured naming convention, (for example Line_Equipment_Signal1, Line_Equipment_Signal2...) you can easily use the influxdb regex functionality to query everything from one equipment at once: select * from /Line_Equipment/ where time > now() - 5m

mirkocomparetti-synesis commented 3 years ago

I get that, but in that way you are increasing the resource consumption, to store a set of data with the same tag, the same timestamp and only one field difference, so like

Signal 1: timestamp, tagA, tagB, tagC, fieldA
Signal 2: timestamp, tagA, tagB, tagC, fieldA
Signal 3: timestamp, tagA, tagB, tagC, fieldA
Signal 4: timestamp, tagA, tagB, tagC, fieldA
Signal 5: timestamp, tagA, tagB, tagC, fieldA
Signal 6: timestamp, tagA, tagB, tagC, fieldA

Instead of

Signals: timestamp, tagA, tagB, tagC, fieldA, fieldB, fieldC, fieldD, fieldE, fieldF

In my opinion, there is a huge difference in the two options: in the second one you have a lot less cardinality of series, even though you can achieve the same result using regexes in the queries.

coussej commented 3 years ago

That is not always true: the OPC-UA status is kept as a tag (not as a field, as you want the status indexed. That was the assumption here, anyway). In your scenario (given that you want the status information to be kept), you'd need a status tag per field.

Say that each sensor randomly returns one of 3 statuses: Good, Bad, Uncertain. When you have three sensors, each in a separate measurement, your cardinality is 3*3 = 9. When you have them under the same measurement, with status tags for each field, your cardinality is 3^3 = 27. This approach could potentially have a much larger impact.

Anyway, if you'd still like the follow that approach, the code is open source and the changes would be rather small, so feel free to do so :-)

mirkocomparetti-synesis commented 3 years ago

Sorry, I didn't explain myself: I'd like to strip out the status tag, i'm not interested in it; the tagA, tagB, tagC are three tags that are custom defined; I was not referring to the quality status, which does not make any sense if you have multiple fields in one entry. Ideally the field should have a "null" value in case the status is not good.

I'll see if I'll manage to look into the code, considering also the new telegraf opcua input plugin.

Thanks