influxdata / telegraf

Agent for collecting, processing, aggregating, and writing metrics, logs, and other arbitrary data.
https://influxdata.com/telegraf
MIT License
14.63k stars 5.58k forks source link

OPC-UA input plugin roadmap #10062

Open sjwang90 opened 2 years ago

sjwang90 commented 2 years ago

Consolidation of OPC-UA plugin issues/feature requests. A lot of bugs and feature requests have arisen since inputs.opcua was released. In this issue would like to consolidate what are library issues vs Telegraf bugs vs plugin feature requests

Background:

Bugs / issues

gopcua Library issues

Feature Requests

Device types

Fanuc - @joschott on community.influxdata.com KUKA DeviceConnector 2 - @mirkocomparetti-synthesis in #9163 OPC Red Lion DA 10 servers - @jnangle server issue addressed in #9807

magiconair commented 2 years ago

Hi, I am one of the main authors and maintainers of the https://github.com/gopcua/opcua library and I am really happy that this library is being used in more areas. I was caught a bit by surprise by the inclusion to influx.

So far we have been using and testing this mostly in our factories at Northvolt with Siemens and Beckhoff PLCs and other maintainers were able to test with a different setup. But we for sure didn't cover the entire OPC/UA landscape with all their edge cases.

Yesterday, we have released the v0.2.0 version of the library which includes auto-reconnect and a cleanup of the subscription code. This was a year in the making since it was a big refactor but we've finally released this and I would like to go back to a faster cadence of smaller releases and bug fixes.

It would be great if we can address the issues you are finding and make the library more stable and useful for a greater community but we need your help since we usually don't have access to the equipment or the software.

magiconair commented 2 years ago

What usually helps to troubleshoot the issues are tcpdumps.

jnangle commented 2 years ago

Hello @magiconair, thank you for finding the Influx group and for providing the updates to the gopcua library. I opened a ticket in that github group for errors involving connections to the Red Lion OPC UA server using numeric identifiers. You had asked for tcpdumps for a working connection, but I haven't had the time to get back my system and run tcpdump. I understand that that is a valuable tool for you, I'll do my best to get you some data today.

magiconair commented 2 years ago

@jnangle Thank you!

sjwang90 commented 2 years ago

@magiconair Thanks for reaching out! Yes this OPC UA plugin is becoming pretty popular with influx users. Here are some things I wanted to run by you.

I'll add the following items to the plugin readme:

Library improvements: Any plugin issues that could be a problem with the gopcua library we'll make sure there's an issue opened in your repo as well.

magiconair commented 2 years ago

At Northvolt we run the code in production in all our factories against Siemens S7-1500 PLCs, Beckhoff PLCs and some Kepware gateways. We are in the process of migrating from v0.1.13 to v0.2.x.

magiconair commented 2 years ago

I'll update the gopcua readme as well. I think this is a good idea.

magiconair commented 2 years ago

https://github.com/gopcua/opcua#tested-platforms

R290 commented 2 years ago

@sjwang90 Thanks for creating this overview! Are you able to find some time to update it?

sjwang90 commented 2 years ago

@R290 Just updated. Let me know if there's anything that's missing. That's for all your PRs for the OPC-UA work.

BSommen commented 2 years ago

@magiconair, any expected timing on adding the "Events / subscription based plugin" feature to the plugin? In our factory we would like to monitor mutiple (200-500) tags and write them to Influx. At the moment we do this with NodeRed, but it fails on the amount of tags he has to write to the DB. I'm hoping that Telegraf and the integration with OPC UA, has a higher performance on this?

While waiting on this new feature, any suggestion to do this?

Jayclifford345 commented 2 years ago

I am just linking two community requests for adding subscriptions. I believe this is quickly elevating to the top of the feature list:

  1. https://community.influxdata.com/t/telegraf-missing-data/25703/3
  2. https://community.influxdata.com/t/telegraf-opc-ua-ingest-and-visualisation-slow/25698
kung-foo commented 2 years ago

@magiconair, any expected timing on adding the "Events / subscription based plugin" feature to the plugin? In our factory we would like to monitor mutiple (200-500) tags and write them to Influx. At the moment we do this with NodeRed, but it fails on the amount of tags he has to write to the DB. I'm hoping that Telegraf and the integration with OPC UA, has a higher performance on this?

While waiting on this new feature, any suggestion to do this?

@BSommen I think you should direct the question to the Telegraf maintainers. @magiconair (and myself) maintain the underlying OPC-UA library (separate from Telegraf), which already supports "subscription" based interfaces (at Intelecy we use it to stream process data with millisecond latency into ClickHouse for up to 40k tags in a single process).

powersj commented 1 year ago

@LarsStegman @R290

Looking for some wisdom and guidance on the remaining four remaining OPC-UA issues listed in the original issue report:

Thanks!

LarsStegman commented 1 year ago
magiconair commented 1 year ago

Re #9911: it is possible to create a new type at runtime (https://pkg.go.dev/reflect#StructOf). Whether it's useful is a different question. It might actually be more useful in the Telegraf context than in our physical factory context.

LarsStegman commented 1 year ago

We might not even need to build the types actually. We might be able to look at the object properties and just log those. Not sure if that will work with the library though.

magiconair commented 1 year ago

If you want to log field values of structured types (extension objects) you need to read them in full first and they need to match the type on the OPC/UA server exactly. The only way to do this with the library is to have a corresponding Go type.