search

FreeOpcUa / python-opcua

LGPL Pure Python OPC-UA Client and Server
http://freeopcua.github.io/
GNU Lesser General Public License v3.0
1.36k stars 658 forks source link

trying to get the children of a node that has lot's of children: Not enough data left in buffer #1181

Open juamar opened 3 years ago

juamar commented 3 years ago

Hi, I 'm trying to get the children of a node that has lot's of children, like this:

node.get_children(refs=33)

I don't know how to determine the number of children it has, but is pretty big. UA Experts takes some minutes to bring all the nodes (which are signals with values) The problem is that when my codes tries to execute that line, i get this Exception:

Not enough data left in buffer, request for 960036864, we have Buffer(size:6523938, data:b'999|PT9801\x03\x05\x.... and it starts printing all the data in the buffer... i suppose.

I have tried to use max_messagesize parameter to see if this increases this buffer, but it's the same.

Is there something i can do to handle this? Maybe there is a way to grt the children in a way that doesn't need to retrieve all the data at once.

BR!

AndreasHeine commented 3 years ago

could you provide some information about the server (vendor, stack, language) and about the variable type

juamar commented 3 years ago

Hi, I only have the configuration name right now: open62541-based OPC UA Application As far as ir remember, this publishes some tags in a SCADA System.

About the Node which is giving problems i have this... The nodeClass is an object. It contains tags, like floats, and ints, and so on. It has type Definition as "StationType", Property "StationNumber", and Components... like this:

image

The components in this case will be the signals. Is this information usefull?

BR!

AndreasHeine commented 3 years ago

could you turn on the logger?

import logging
logging.basicConfig(level=logging.DEBUG)

and post the browse console output! a stacktrace would also be helpfull

juamar commented 3 years ago

Okey, for the sake of simplicity i did this code to specially test this situation:

import logging
from opcua import Client
from NodeXmlExporter import NodeXMLExporter

logging.basicConfig(filename="trace.log",level=logging.DEBUG)

exporter = NodeXMLExporter()
exporter.connect("opc.tcp://50.200.105.156:4840")
nodeid="ns=1;s=99999|DASMAP"
node = exporter.client.get_node(nodeid)
node.get_children(refs=33)
exporter.client.disconnect()

Now, the exporter connect goes like this:

import argparse
import logging
import sys
import time
from opcua import Client
from XmlExporter import XmlExporter

class NodeXMLExporter:
    def __init__(self):
        self.nodes = []
        self.namespaces = {}
        self.visited = []
        self.client = None
        self.logger = logging.getLogger(__name__)
        #self.logger.setLevel(logging.INFO)

    ...

    def connect(self, server_url="opc.tcp://localhost:16664"):
        from opcua.crypto import security_policies
        import types
        from opcua.ua.uaprotocol_hand import CryptographyNone

        self.client = Client(server_url)
        self.client.session_timeout = 3600000

        # Fix symmetric_key_size (not 0) of securityPolicy
        sec_policy = security_policies.SecurityPolicy()
        sec_policy.symmetric_key_size = 8
        self.client.security_policy = sec_policy

        # Fix signature method of CryptographyNone
        def signature(self, data):
            return None
        fixed_signature = types.MethodType(signature, CryptographyNone)
        self.client.security_policy.asymmetric_cryptography.signature = fixed_signature
        self.client.max_messagesize = 99999999

        try:
            self.client.connect()
            self.client.max_messagesize = 99999999
            self.client.session_timeout = 3600000
        except Exception as e:
            self.logger.error("No connection established", e)
            self.logger.error(e)
            self.logger.error("Exiting ...")
            sys.exit()

        self.logger.info("Client connected to %s" % server_url)
        self.logger.debug("Timeout = %d" % self.client.session_timeout)

The output i can retrieve is this:

INFO:opcua.client.ua_client.Socket:opening connection
INFO:opcua.client.ua_client.Socket:Thread started
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'ACK', chunk_type:b'F', body_size:20, channel:0)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.client.ua_client.Socket:open_secure_channel
DEBUG:opcua.client.ua_client.Socket:Sending: OpenSecureChannelRequest(TypeId:{self.TypeId}, RequestHeader:{self.RequestHeader}, Parameters:{self.Parameters})
INFO:opcua.uaprotocol:received header: Header(type:b'OPN', chunk_type:b'F', body_size:123, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
WARNING:opcua.client.client:Requested secure channel timeout to be 3600000ms, got 600000ms instead
INFO:opcua.client.ua_client:create_session
DEBUG:opcua.client.ua_client.Socket:Sending: CreateSessionRequest(TypeId:{self.TypeId}, RequestHeader:{self.RequestHeader}, Parameters:{self.Parameters})
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'F', body_size:571, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
DEBUG:opcua.client.ua_client:CreateSessionResponse(TypeId:{self.TypeId}, ResponseHeader:{self.ResponseHeader}, Parameters:{self.Parameters})
DEBUG:opcua.client.client:starting keepalive thread with period of 420000.0 milliseconds
INFO:opcua.client.ua_client:activate_session
DEBUG:opcua.client.ua_client.Socket:Sending: ActivateSessionRequest(TypeId:{self.TypeId}, RequestHeader:{self.RequestHeader}, Parameters:{self.Parameters})
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'F', body_size:84, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
DEBUG:opcua.client.ua_client:ActivateSessionResponse(TypeId:{self.TypeId}, ResponseHeader:{self.ResponseHeader}, Parameters:{self.Parameters})
INFO:NodeXmlExporter:Client connected to opc.tcp://50.200.105.156:4840
DEBUG:NodeXmlExporter:Timeout = 3600000
INFO:opcua.client.ua_client:browse
DEBUG:opcua.client.ua_client.Socket:Sending: BrowseRequest(TypeId:{self.TypeId}, RequestHeader:{self.RequestHeader}, Parameters:{self.Parameters})
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65523, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65523, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65523, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65523, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65523, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65523, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65523, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65523, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65520, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65523, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65523, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65523, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65523, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65523, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65523, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65523, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65523, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65523, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65523, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65523, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65523, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65523, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65523, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65523, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65523, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65523, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65523, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65523, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65523, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65523, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65523, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65520, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'C', body_size:65522, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header
INFO:opcua.uaprotocol:received header: Header(type:b'MSG', chunk_type:b'F', body_size:38447, channel:3)
DEBUG:opcua.uaprotocol:Waiting for header

And the the program starts printing the buffer error i mention in the beggining, which is so big it overflows the console buffer, and i cannot get it as a hole unless i do i try catch of the error and tell the code to print it in some file, as i was doing earlier with the logging library. As i said before, the error goes like this:

Not enough data left in buffer, request for 960036864, we have Buffer(size:6523938, data:b'999|PT9801\x03\x05\x.... and it starts printing all the data in the buffer... i suppose.

I saw yesterday that raise that prints this error is located in a Buffer class within your library.

Thanks for your help!

AndreasHeine commented 3 years ago

ok

max_messagesize = 99999999 -> the real max messagesize is negotiated between client and server during the Hello/Ack handshake on connect https://github.com/FreeOpcUa/python-opcua/blob/131ad8ceffeb7510b7bb3d9db1742896dbf74196/opcua/client/ua_client.py#L180

means if the server does not support "99999999" the actually used max messagesize might be lower! if you want to check that you can use wireshark and trace the connect filter "opcua" and you can check "hello" and the response "ack" the smaller messagesize will be used during the connection

so the main question is now what is the used messagesize?

Referring to: https://reference.opcfoundation.org/v104/Core/docs/Part6/7.1.2/

juamar commented 3 years ago

Okeeyy, interesting.

This is the Hello: image

And this the acknowledge: image

I think maybe the problem is not in the Message Size, but in the buffer size... but the error says the buffer size is different.

juamar commented 3 years ago

I'm trying to analize the response for the browse the code is doing internally. this is the result:

image

The hole Frame weights 7179051 bytes, which is 7.718 MB But the code is now telling me this (only on console):

Traceback (most recent call last): File "d:\javendano\windows\Codes\opc_ua_xml_export_client\opcuaTeamHelp.py", line 11, in node.get_children(refs=33) File "C:\Users\javendano\AppData\Roaming\Python\Python39\site-packages\opcua\common\node.py", line 316, in get_children return self.get_referenced_nodes(refs, ua.BrowseDirection.Forward, nodeclassmask) File "C:\Users\javendano\AppData\Roaming\Python\Python39\site-packages\opcua\common\node.py", line 390, in get_referenced_nodes references = self.get_references(refs, direction, nodeclassmask, includesubtypes) File "C:\Users\javendano\AppData\Roaming\Python\Python39\site-packages\opcua\common\node.py", line 369, in get_references results = self.server.browse(params) File "C:\Users\javendano\AppData\Roaming\Python\Python39\site-packages\opcua\client\ua_client.py", line 327, in browse data = self._uasocket.send_request(request) File "C:\Users\javendano\AppData\Roaming\Python\Python39\site-packages\opcua\client\ua_client.py", line 83, in send_request data = future.result(self.timeout) File "C:\Program Files\Python39\lib\concurrent\futures_base.py", line 442, in result raise TimeoutError() concurrent.futures._base.TimeoutError

Previous executions where made on linux, but this one is being made on windows. Looks like the error is the same, but on windows Python will return timeout, and on linux the real error.

juamar commented 3 years ago

Some update. If i set self.client.secure_channel_timeout = 3600000 as a connection parameter, the code will return the same error as on linux. Messages on Wireshark are exactly the same.

juamar commented 3 years ago

Also, it seems like UAExpert uses BrowseNext for the task instead of Browse. UAExpert is able to bring all the references to the components of the Node.

AndreasHeine commented 3 years ago

get_children uses browse_next aswell! https://github.com/FreeOpcUa/python-opcua/blob/131ad8ceffeb7510b7bb3d9db1742896dbf74196/opcua/common/node.py#L294 https://github.com/FreeOpcUa/python-opcua/blob/131ad8ceffeb7510b7bb3d9db1742896dbf74196/opcua/common/node.py#L384 https://github.com/FreeOpcUa/python-opcua/blob/131ad8ceffeb7510b7bb3d9db1742896dbf74196/opcua/common/node.py#L348

https://github.com/FreeOpcUa/python-opcua/blob/131ad8ceffeb7510b7bb3d9db1742896dbf74196/opcua/client/ua_client.py#L333

AndreasHeine commented 3 years ago

could you post the server capabilities?

like this: 1234211

juamar commented 3 years ago

I don't understand much about OPCUA and the protocol, but the Python is doing the request like this:

image

And OPC UA something like this (much longer i have to say):

image

These are the server capabilities:

image

AndreasHeine commented 3 years ago

::1 = IPv6 Localhost 127.0.0.1 = IPv4 Localhost

juamar commented 3 years ago

With IP4 is the same result as with IP6, and it executes the same way.

zerox1212 commented 3 years ago

Can you change the server? Is 8mb of children really a smart design?

starturtle commented 3 years ago

Does the parent node have huge children, or are there indeed very many? How about limiting the max number of references to return? If I remember correctly, that part can be tweaked on both sides. In that case it should be possible to stream and process the result list in chunks.

AndreasHeine commented 3 years ago

thats actually what browse next does! but i am not sure if 8mb is possible with <8mb buffer 🤔 because i cant construct the message in one shot

juamar commented 3 years ago

@starturtle There are very many. The children are just variables.

juamar commented 3 years ago

I am encountering the same problem using another library. As i try to find a solution, i will see if there is a chance of dividing that node in many nodes with less children (or group by inner nodes).

AndreasHeine commented 3 years ago

or you need to configure the server with a bigger buffer! what in my opinion is a serverside error is that the server does not return a statuscode he just sends not all data... the buffer should always be as big as the biggest message it can send!

juamar commented 3 years ago

Finally i am using a .NET library for the pourpose we where trying to achieve. The solution would be to limit in some way how many nodes we retrieve per request. I insist, the solution i applied was to do first a browse request with, ie 100 nodes, and then the subsequent request to do it with browseNext. When i do 

node.get_children()
with you library it does a browse of everything, and then it crashes, because the task is to big for a single browse.

Maybe if i have seen a lower level example of how to retrieve data from OPC.UA, i would have achieved the task with this library.

It is also true that my datasource has to many useless nodes, and we are trying to tell the admins of that datasource to drop them in some way. Nevertheless, it is possible to reach the data :)

Thanks for your help guys. Better than paid support!

starturtle commented 3 years ago

Wouldn't it help with such issues if node.get_references provided an interface to set RequestedMaxReferencesPerNode on the browse parameters? That may enable a client on slow hardware to run against a server designed for the greedy. Or am I confusing arguments here?

oroulet commented 3 years ago

Looking at code of get_chilfren() and using the low level api you can do whatever you want. It would have been an easy solution.

But i am interested to know how that parsing error happened. There is s bug somewhere

juamar commented 3 years ago

Wouldn't it help with such issues if node.get_references provided an interface to set RequestedMaxReferencesPerNode on the browse parameters? That may enable a client on slow hardware to run against a server designed for the greedy. Or am I confusing arguments here?

Very Greedy like mine. Later last month I knew that the OPC UA Server was done with some C or C++ code kind of fast and quick, by someone with not much time on the server.... neither in his priorities XD

juamar commented 3 years ago

So cool the commits. Will be nice to test its functionality

AndreasHeine commented 3 years ago

adding MaxReferencesPerNode to the api makes sens if a client checks the servers operational limits up front! but it dont solve the problem if you have more then +10k subnodes it only works till you reach maxmessagesize then its game over... if a server/client supports a gigantic messagesize you need to bump up timeouts tested it with 100k Children (client.uaclient._timeout = 10) Unbenannt

but it looks like we never check if a reponse with more then negotiated messagesizes should ever been send out

zerox1212 commented 3 years ago

but it looks like we never check if a reponse with more then negotiated messagesizes should ever been send out

Should probably at lease be a log warning here...