search

influxdata / telegraf

Agent for collecting, processing, aggregating, and writing metrics, logs, and other arbitrary data.
https://influxdata.com/telegraf
MIT License
14.63k stars 5.58k forks source link

[opcua_listener] 1000 Nodes will get EOF[failed to start monitoring items: EOF] #14568

Closed yang750418 closed 9 months ago

yang750418 commented 9 months ago

Relevant telegraf.conf

.....
[[inputs.opcua_listener]]
  name = "Mapping"
  endpoint = "opc.tcp://192.168.1.xx" 
  connect_timeout = "100s"
  request_timeout = "50s"
  subscription_interval = "100ms"
  security_policy = "auto"
  security_mode = "auto"
  certificate = "/etc/telegraf/ua.pem"
  private_key = "/etc/telegraf/ua_key.pem"
  auth_method = "UserName"
  username = "user"
  password = "123456"
  timestamp = "gather"
......

Logs from Telegraf

failed to start monitoring items: EOF

System info

Telegraf 1.29.0 ,docker 24.0.2, IGS OPCUA

Docker

No response

Steps to reproduce

  1. Setting security_policy=auto & security_mode=auto
  2. Setting certificate & private_key
  3. Config 2000~3000 Nodes
  4. Run Telegraf Connection OPC Ua ...

Expected behavior

start monitoring items success

Actual behavior

failed to start monitoring items: EOF

Additional info

Telegraf Logs 292698453-edb572ff-e1de-42f4-ac8f-0cd3ebb23b4f When I cancel the use of security settings, I can configure 20000 NodeID.

powersj commented 9 months ago

Hi,

Looking at the upstream gopcua library it seems other users have reported similar issues:

https://github.com/gopcua/opcua/issues/531 https://github.com/gopcua/opcua/issues/658 https://github.com/gopcua/opcua/issues/698

Does connecting to a single node with security settings work? That would rule out any misconfiguring there.

yang750418 commented 9 months ago

Hi powersj

Connecting to a single node with security settings works fine. Using security settings can also work normally when the number of nodes is 900. It seems that the encrypted length exceeds the limit.

Thanks

powersj commented 9 months ago

@yang750418,

I have put up https://github.com/influxdata/telegraf/pull/14595 which enables the opcua client debug logger. can you please download an artifact from that PR and run with it? Please provide the collected logs.

Thanks

edit:

You will need to set two things to get the additional messages:

1 - debug in the agent via --debug CLI option or adding debug=true to the [agent] section in your config. 2 - set client_trace = true in your OPC UA config

yang750418 commented 9 months ago 
2024-01-19T14:57:41+08:00 I! Starting Telegraf 1.30.0-2e24e185 brought to you by InfluxData the makers of InfluxDB
2024-01-19T14:57:41+08:00 I! Available plugins: 241 inputs, 9 aggregators, 30 processors, 24 parsers, 60 outputs, 6 secret-stores
2024-01-19T14:57:41+08:00 I! Loaded inputs: opcua_listener
2024-01-19T14:57:41+08:00 I! Loaded aggregators: 
2024-01-19T14:57:41+08:00 I! Loaded processors: 
2024-01-19T14:57:41+08:00 I! Loaded secretstores: 
2024-01-19T14:57:41+08:00 I! Loaded outputs: influxdb_v2 kafka
2024-01-19T14:57:41+08:00 I! Tags enabled: host=966c274afa65
2024-01-19T14:57:41+08:00 I! [agent] Config: Interval:500ms, Quiet:false, Hostname:"966c274afa65", Flush Interval:1s
2024-01-19T14:57:41+08:00 D! [agent] Initializing plugins
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] Initialising OpcUAInputClient
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] Initialising OpcUAClient
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] Initialising node to metric mapping
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] Creating monitored items
2024-01-19T14:57:41+08:00 D! [agent] Connecting outputs
2024-01-19T14:57:41+08:00 D! [agent] Attempting connection to [outputs.influxdb_v2]
2024-01-19T14:57:41+08:00 D! [agent] Successfully connected to outputs.influxdb_v2
2024-01-19T14:57:41+08:00 D! [agent] Attempting connection to [outputs.kafka]
2024-01-19T14:57:41+08:00 D! [sarama] Initializing new client
2024-01-19T14:57:41+08:00 D! [sarama] client/metadata fetching metadata for all topics from broker XXX:29092
2024-01-19T14:57:41+08:00 D! [sarama] SASL authentication succeeded
2024-01-19T14:57:41+08:00 D! [sarama] Connected to broker at XXX:29092 (unregistered)
2024-01-19T14:57:41+08:00 D! [sarama] client/brokers registered new broker #1036 at XXX:29092
2024-01-19T14:57:41+08:00 D! [sarama] Successfully initialized new client
2024-01-19T14:57:41+08:00 D! [agent] Successfully connected to outputs.kafka
2024-01-19T14:57:41+08:00 D! [agent] Starting service inputs
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] Connecting OPC UA Client to server
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uacp: connecting to opc.tcp://192.168.1.XX:48050
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uacp 1: start HEL/ACK handshake
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uacp 1: sent HELF with 60 bytes
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uacp 1: recv ACKF with 28 bytes
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uacp 1: recv &uacp.Acknowledge{Version:0x0, ReceiveBufSize:0xffff, SendBufSize:0xffff, MaxMessageSize:0xffff00, MaxChunkCount:0x100}
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uasc 1/1: send *ua.OpenSecureChannelRequest with 132 bytes
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uacp 1: recv OPNF with 136 bytes
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uasc 1/1: recv OPNF with 136 bytes
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uasc 1/1: recv *ua.OpenSecureChannelResponse
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uasc 1/1: sending *ua.OpenSecureChannelResponse to handler
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uasc 1: received security token. channelID=2437948123 tokenID=1 createdAt=2024-01-19T06:57:35Z lifetime=1h0m0s
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uasc 1: security token is refreshed at 2024-01-19T07:42:41Z (45m0s). channelID=2437948123 tokenID=1
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uasc 1: security token expires at 2024-01-19T08:12:35Z. channelID=2437948123 tokenID=1
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uasc 1/2: send *ua.GetEndpointsRequest with 97 bytes
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uacp 1: recv MSGF with 3690 bytes
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uasc 1/2: recv MSGF with 3690 bytes
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uasc 1/2: recv *ua.GetEndpointsResponse
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uasc 1/2: sending *ua.GetEndpointsResponse to handler
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uasc 1: Close()
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uasc 1/3: send *ua.CloseSecureChannelRequest with 57 bytes
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uacp 1: close
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uasc 1: readChunk EOF
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] Configuring OPC UA connection options
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] Loading cert/key from /etc/telegraf/ua.pem//etc/telegraf/ua_key.pem
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] security policy from configuration 
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uacp: connecting to opc.tcp://192.168.1.XX:48050
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uacp 2: start HEL/ACK handshake
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uacp 2: sent HELF with 60 bytes
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uacp 2: recv ACKF with 28 bytes
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uacp 2: recv &uacp.Acknowledge{Version:0x0, ReceiveBufSize:0xffff, SendBufSize:0xffff, MaxMessageSize:0xffff00, MaxChunkCount:0x100}
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uasc 2/1: send *ua.OpenSecureChannelRequest with 1905 bytes
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uacp 2: recv OPNF with 1901 bytes
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uasc 2: setting securityPolicy to http://opcfoundation.org/UA/SecurityPolicy#Basic256Sha256
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uasc 2/1: recv OPNF with 1901 bytes
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uasc 2/1: recv *ua.OpenSecureChannelResponse
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uasc 2/1: sending *ua.OpenSecureChannelResponse to handler
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uasc 2: received security token. channelID=2437948124 tokenID=1 createdAt=2024-01-19T06:57:35Z lifetime=1h0m0s
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uasc 2: security token is refreshed at 2024-01-19T07:42:41Z (45m0s). channelID=2437948124 tokenID=1
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uasc 2: security token expires at 2024-01-19T08:12:35Z. channelID=2437948124 tokenID=1
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uasc 2/2: send *ua.CreateSessionRequest with 1584 bytes
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uacp 2: recv MSGF with 5408 bytes
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uasc 2/2: recv MSGF with 5408 bytes
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uasc 2/2: recv *ua.CreateSessionResponse
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uasc 2/2: sending *ua.CreateSessionResponse to handler
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uasc 2/3: send *ua.ActivateSessionRequest with 464 bytes
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uacp 2: recv MSGF with 144 bytes
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uasc 2/3: recv MSGF with 144 bytes
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uasc 2/3: recv *ua.ActivateSessionResponse
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uasc 2/3: sending *ua.ActivateSessionResponse to handler
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uasc 2/4: send *ua.ReadRequest with 144 bytes
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uacp 2: recv MSGF with 480 bytes
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uasc 2/4: recv MSGF with 480 bytes
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uasc 2/4: recv *ua.ReadResponse
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uasc 2/4: sending *ua.ReadResponse to handler
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] Connected to OPC UA Server
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] Creating OPC UA subscription
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uasc 2/5: send *ua.CreateSubscriptionRequest with 128 bytes
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uacp 2: recv MSGF with 112 bytes
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uasc 2/5: recv MSGF with 112 bytes
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uasc 2/5: recv *ua.CreateSubscriptionResponse
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uasc 2/5: sending *ua.CreateSubscriptionResponse to handler
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] Subscribed with subscription ID 2432947172
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uasc 2/6: send *ua.CreateMonitoredItemsRequest with 65536 bytes
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uasc 2/6: send *ua.CreateMonitoredItemsRequest with 20544 bytes
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uasc 2/7: send *ua.PublishRequest with 112 bytes
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uasc 2: readChunk EOF
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uacp 2: close
2024-01-19T14:57:41+08:00 D! [inputs.opcua_listener] uasc 2: Close()
2024-01-19T14:57:41+08:00 E! [telegraf] Error running agent: starting input inputs.opcua_listener: failed to start monitoring items: EOF

@powersj The above is the telegraf log. The Node Name was shortened and 1500 Nodes were configured, but failed.

Thanks

powersj commented 9 months ago

Nothing there stands out to me, but I'm not an OPCUA expert. Please create a discussion with the gopcua upstream project and ask there.

telegraf-tiger[bot] commented 9 months ago

Hello! I am closing this issue due to inactivity. I hope you were able to resolve your problem, if not please try posting this question in our Community Slack or Community Forums or provide additional details in this issue and reqeust that it be re-opened. Thank you!