Multiple Subscriptions

[tdesroches]

I'm relatively new to python and opcua, I've used this to make a server and client and it's working well. However I'm quite certain that I'm not doing it efficiently. Essentially what I'm doing is passing a list of nodes to subscribe_data_change. This works however when the event handler is fired I then have to parse out what the node it was that changed then do what I want with the value. To see my terrible code see https://github.com/tdesroches/Dataminer/blob/master/DataMinerMain.py.

Any suggestion anyone may have please comment. I'm just wondering if there is a better way and perhaps a little example.

[zerox1212]

You should look at how the handler is done in this example: https://github.com/FreeOpcUa/python-opcua/blob/master/examples/server-ua-python-mirror.py

This example makes minor edits to do what you want. The key difference is that the SubHandler class has a constructor with the object argument. This will give the sub handler a reference to it's parent object. No need to parse anything manually. (Note you will need to capture a lot of that data you have copy pasted many times into a single class)

class SubHandler(object):
    """
    Subscription Handler. To receive events from server for a subscription.
    The handler forwards updates to it's referenced python object
    """

    def __init__(self, obj):
        self.obj = obj

Probably the best way would be to further follow the pattern of that example, where your custom object has a function like insert_sql(). Then you can do this in the sub handler:

def datachange_notification(self, node, val, data):
        # print("Python: New data change event", node, val, data)

        _node_name = node.get_browse_name()
        setattr(self.obj, _node_name.Name, data.monitored_item.Value.Value.Value) #this sets the value in your python object

        self.obj.calc_and_insert_sql(val)

There are other options and for people that are new to Object Oriented Programming it's often a struggle with how to best handle subscriptions (I did anyways).

In my opinion this system could use some work, because I feel like it's too difficult to relate the datachange_notification to non OPC UA objects. It basically forces you to do some kind of look up based on the node field. Maybe in the future we can do a more straightforward "call back" design.

[zerox1212]

I had some free time, so here is an example of what your object might look like.

HMIValue(object):

    def __init__(self, opcua_server, ua_node number_of_x, table_name):
        self.ua_node = ua_node
        self.number_of_x = number_of_x
        self.table_name = table_name
        self.lastPosX = 0
        self.lastPosY = 0

        # subscribe to ua node of this python object (add 'self' if you want to keep track of these objects)
        handler = SubHandler(self)
        sub = opcua_server.create_subscription(500, handler)
        handle = sub.subscribe_data_change(self.ua_node)

    def calc_and_insert_sql(self, val):
         if (abs(self.lastPosX[axis]-float(val[axis]))) > 1:
                    print(self.lastPosX[axis],val[axis])
                    print('X{axis} position = {val}'.format(axis=axis+1,val=val[axis]))
                    query = "INSERT INTO {table} (motor, position) VALUES('{val1}', '{val2}')".format(val1="X" + str(axis + 1),
                                                                                                      val2=val[axis],
                                                                                                      table=self.tablename)

Then create your HMI object:

grHMI_XActualPosition = HMIValue(my_server, my_node, 2, 'motor_position')

Note this is not a fully working example, but should give you an idea how to do it.

[tdesroches]

That's fantasic! I knew I was doing it the hard way. Thanks I will try it.

[tdesroches]

Hey Zerox,

I've tried implementing you suggestion in a simple client, I must be missing something. When I run it I get concurrent.future timeout errors. Below is my attempt. I stripped it down just so I can wrap my head around how it works.

import sys
import time
sys.path.insert(0, "..")
from opcua import Client
class SubHandler(object):
    def __init__(self, obj):
        self.obj = obj
    def datachange_notification(self, node, val, data):
        _node_name = node.get_browse_name()
        setattr(self.obj, _node_name.Name, data.monitored_item.Value.Value.Value)
        print("Python: New data change event", node, val)

class value(object):
    def __init__(self, opcua_server, ua_node):
        self.ua_node = ua_node
        handler = SubHandler(self)
        sub = opcua_server.create_subscription(500, handler)
        handle = sub.subscribe_data_change(self.ua_node)

if __name__ == "__main__":
    client = Client("opc.tcp://localhost:4840/freeopcua/server/")
    try:
        client.connect()
        root = client.get_root_node()
        myvar = root.get_child(["0:Objects", "2:MyObject", "2:MyVariable"])
        variableObj = value(client, myvar)
        time.sleep(10)
    finally:
        client.disconnect()

[tdesroches]

So part of the issue is for some reason it doesn't like the .get_browse_name(). If I remove this an simple print val then the subscription works.

[oroulet]

Never do a network operation in a handler. It will block! This is documented many places

[tdesroches]

So I have it sort of working. With a print statement in the datachange_notification method I know the sub is working. Why the get_browse_name() doesn't work I'm not sure. I'm just having trouble now pulling the value from the object see the code below.

import sys
import time
sys.path.insert(0, "..")

from opcua import Client

class SubHandler(object):
    def __init__(self, obj):
        self.obj = obj

    def datachange_notification(self, node, val, data):
        print("Python: New data change event", node, val, data)
        #_node_name = node.get_browse_name() <---get_browse_name() not working
        setattr(self.obj, 'myvar', data.monitored_item.Value.Value.Value)

class value(object):

    def __init__(self, opcua_server, ua_node):
        self.ua_node = ua_node
        self.MyVariable = 0

        handler = SubHandler(self)
        sub = opcua_server.create_subscription(500, handler)
        handle = sub.subscribe_data_change(self.ua_node)
        super().__init__()

if __name__ == "__main__":
    client = Client("opc.tcp://localhost:4840/freeopcua/server/")
    try:
        client.connect()
        root = client.get_root_node()
        objNode = client.get_objects_node()
        myvar = root.get_child(["0:Objects", "2:MyObject", "2:MyVariable"])
        variableObj = value(client, myvar)
        print(getattr(variableObj,'myvar'))
        time.sleep(10)

    finally:
        client.disconnect()

[tdesroches]

Thanks oroulet I was thinking if I can get this subscription working I will pull the network operation out of the event hander and do a seperate thread periodically to update the db.

[zerox1212]

Sorry for the confusion, my example was more geared toward server side. (and I never had issues with doing get browsename on server side, should I not do this???)

@oroulet can you post a better design for subhandler? I can't see a clear way to use SubHandler inside an arbitrary python class. You always have to look up what node has called datachange_notification. I'm still confused on how best to use this design...

[zerox1212]

@tdesroches getbrowsename() doesn't work from inside datachange_notification for you because that datachange function is called by the network thread, so making a subsequent network call (blocking) stops all other communication.

The example I gave you is for server code, which I guess doesn't block long enough to actually cause an issue, or the magic asyncio library is handling it. The client library doesn't use asyncio.

Secondly, you need to look up the docs for how setattr works. setattr(self.obj, 'myvar', data.monitored_item.Value.Value.Value) should be updating 'MyVariable', not 'myvar'. Your current code is creating a second class attribute named 'myvar'.

Also you do not need to make a call to super() because you are not sub-classing a custom object (like the example is).

[tdesroches]

@zerox1212 Ah of course that makes sense thank you

[zerox1212]

I guess a possible work around would be to store the browsename in the class itself. Then reference it in SubHandler.

class value(object):

    def __init__(self, opcua_server, ua_node):
        self.ua_node = ua_node
        self.browse_name = ua_node.get_browse_name()
        self.MyVariable = 0

        handler = SubHandler(self)
        sub = opcua_server.create_subscription(500, handler)
        handle = sub.subscribe_data_change(self.ua_node)

Then update handler like this.

class SubHandler(object):
    def __init__(self, obj):
        self.obj = obj

    def datachange_notification(self, node, val, data):
        print("Python: New data change event", node, val, data)
        #remove network operation
        setattr(self.obj, self.browse_name, data.monitored_item.Value.Value.Value) # add self.browse_name attr lookup

If you want to test something, you could override __setattr__ in your custom class and see if you can do your SQL stuff there, but I don't know what thread will call setattr. It might still block the network thread. :(

Add this to your value class and find out.

    def __setattr__(self, key, value):
       if key == 'MyVariable':
            insert_SQL()

[tdesroches]

I've added the override to the value class however it cause an AttributeError: 'value' object has no attribute 'ua_node'. I may have not done it correctly. In the SubHandler I wasn't able to get the setattr(self.obj, **self.browse_name**, data.monitored_item.Value.Value.Value) working. I complained about the position argument **self.browse_name** unless that was just to draw my attention to the change lol.

import sys
import time
sys.path.insert(0, "..")

from opcua import Client

class SubHandler(object):
    def __init__(self, obj):
        self.obj = obj

    def datachange_notification(self, node, val, data):
        print("Python: New data change event", node, val, data)
        setattr(self.obj, 'myvar', data.monitored_item.Value.Value.Value)
class value(object):

    def __setattr__(self, key, value):
        print(key, value)

    def __init__(self, opcua_server, ua_node):
        self.ua_node = ua_node
        self.MyVariable = 0
        self.browse_name = ua_node.get_browse_name()

        handler = SubHandler(self)
        sub = opcua_server.create_subscription(500, handler)
        handle = sub.subscribe_data_change(self.ua_node)
        super().__init__()

if __name__ == "__main__":
    client = Client("opc.tcp://localhost:4840/freeopcua/server/")
    try:
        client.connect()
        root = client.get_root_node()
        objNode = client.get_objects_node()
        myvar = root.get_child(["0:Objects", "2:MyObject", "2:MyVariable"])
        variableObj = value(client, myvar)
        print(variableObj.browse_name)
        time.sleep(10)

    finally:
        client.disconnect()

And the output

ua_node Node(NumericNodeId(ns=2;i=2))
MyVariable 0
browse_name QualifiedName(2:MyVariable)
Traceback (most recent call last):
  File "/home/myrddin/PycharmProjects/freeOPCUA/Client.py", line 37, in <module>
    variableObj = value(client, myvar)
  File "/home/myrddin/PycharmProjects/freeOPCUA/Client.py", line 26, in __init__
    handle = sub.subscribe_data_change(self.ua_node)
AttributeError: 'value' object has no attribute 'ua_node'

Process finished with exit code 1

[zerox1212]

The error says your value class (named "self" in your code) has no attribute ua_node. I think there is something else you need there besides print because you are overriding a builtin.

And yes, remove the **, I deleted them from my example.

EDIT: You need to call default behaviour when overridding __setattr__.

def __setattr__(self, key, value):
        super().__setattr__(name, value) #call default behavior and actually set the value of a class attr
        print(key, value)

If this doesn't work you will have to create some other system to do your SQL inserts, and your subscription handling will only be in charge of updating the variable in your value class. Perhaps you can make a queue system where the subscription just adds a value to the queue, and another part of your code can insert to DB everything in the queue every 5 seconds or something.

[tdesroches]

It seems to be related to the setattr override. If I remove to override than I'm able to access ua_node. The line self.ua_node = ua_node in the init of the value object should be creating the attribute i think. Could overriding the setattr be breaking this?

[tdesroches]

@zerox1212 also in your example I made a slight change to the setattr in the handler self.browse_name = ua_node.get_browse_name().Name otherwise it was complaining about not being a string

[oroulet]

You really need to know what you are doing when using setattr you may get funny results ;-)

On Mon, Oct 23, 2017, 19:40 tdesroches notifications@github.com wrote:

@zerox1212 https://github.com/zerox1212 also in your example I made a slight change to the setattr in the handler self.browse_name = ua_node.get_browse_name().Name otherwise it was complaining about not being a string

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/FreeOpcUa/python-opcua/issues/502#issuecomment-338739132, or mute the thread https://github.com/notifications/unsubscribe-auth/ACcfzre3bzCh4rSSLUqEKH9VM6chM8GOks5svM-igaJpZM4QCHwH .

[tdesroches]

@zerox1212 I saw your edit and I have it working now, mostly. I think all that is needed now is to get the value assigned to the attribute. I'm not sure how to do this. If I use print(getattr(variableObj, "MyVariable")) Then it looks like its returning an object rather than the value of MyVariable

import sys
import time
sys.path.insert(0, "..")

from opcua import Client

class SubHandler(object):
    def __init__(self, obj):
        self.obj = obj

    def datachange_notification(self, node, val, data):
        setattr(self.obj, self.obj.browse_name, data.monitored_item.Value.Value.Value)

class value(object):

    def __setattr__(self, key, value):
        print(key, value)

    def __init__(self, opcua_server, ua_node):

        super().__setattr__("ua_node",ua_node)
        super().__setattr__("browse_name",ua_node.get_browse_name().Name)
        super().__setattr__("MyVariable", value)

        handler = SubHandler(self)
        sub = opcua_server.create_subscription(500, handler)
        handle = sub.subscribe_data_change(self.ua_node)
        super().__init__()

if __name__ == "__main__":
    client = Client("opc.tcp://localhost:4840/freeopcua/server/")
    try:
        client.connect()
        root = client.get_root_node()
        objNode = client.get_objects_node()
        myvar = root.get_child(["0:Objects", "2:MyObject", "2:MyVariable"])
        variableObj = value(client, myvar)
        print(getattr(variableObj, "MyVariable"))
        time.sleep(10)

    finally:
        client.disconnect()

[tdesroches]

I think I have it, does it seem about right?

class value(object):

    def __setattr__(self, key, value):
        super().__setattr__("MyVariable", value)
        print(key, value)

    def __init__(self, opcua_server, ua_node):

        super().__setattr__("ua_node",ua_node)
        super().__setattr__("browse_name",ua_node.get_browse_name().Name)
        self.MyVariable = 0
        handler = SubHandler(self)
        sub = opcua_server.create_subscription(500, handler)
        handle = sub.subscribe_data_change(self.ua_node)
        super().__init__()

[zerox1212]

The code using setattr is already saving the value from OPC UA to the python attribute. You just need to do print(variableObj.MyVariable)

Also you are calling super() for setattr wrong. Try this code:

import sys
import time
sys.path.insert(0, "..")

from opcua import Client

class SubHandler(object):
    def __init__(self, obj):
        self.obj = obj

    def datachange_notification(self, node, val, data):
        setattr(self.obj, self.obj.browse_name, data.monitored_item.Value.Value.Value)

class value(object):

    def __setattr__(self, key, value):
        print(key, value)
        super().__setattr__(key, value)

    def __init__(self, opcua_server, ua_node):
        self.ua_node = ua_node
        self.MyVariable = 0
        self.browse_name = ua_node.get_browse_name().Name

        handler = SubHandler(self)
        sub = opcua_server.create_subscription(500, handler)
        handle = sub.subscribe_data_change(self.ua_node)

if __name__ == "__main__":
    client = Client("opc.tcp://localhost:4840/freeopcua/server/")
    try:
        client.connect()
        root = client.get_root_node()
        objNode = client.get_objects_node()
        myvar = root.get_child(["0:Objects", "2:MyObject", "2:MyVariable"])
        variableObj = value(client, myvar)
        print(variableObj.MyVariable)
        time.sleep(10)

    finally:
        client.disconnect()

You have to understand that a OPC UA object (like a node) and a python Object are completely different things. The example we are working on we are trying to synchronize OPC UA data with Python data. That way you only need to work with Python and you don't need to do so many OPC UA operations.

[zerox1212]

@oroulet You are a better programmer than me. How should datachange_notification be used on the client if you can't do any work (that takes time) there because it is blocking network thread?

[tdesroches]

I was at least in the ballpark lol. i made adjustments as per your example and it is working. This has given me plenty that I need to research and learn. Thank you guys for your help. My first time posting on GitHub and great response. :)

I will also pull the network operations out of the handler as documented.

[tdesroches]

I was thinking about firing up a thread but I haven't tried it

[zerox1212]

If you don't need to guarantee your data is 100% up to date when you write to SQL you don't need a thread. You can do SQL writes on the main thread, the UA subscription is already on it's own thread. With the code above you could extend the value object to have a function named write_to_sql and just do something like this:

if __name__ == "__main__":
    client = Client("opc.tcp://localhost:4840/freeopcua/server/")
    try:
        client.connect()
        root = client.get_root_node()
        objNode = client.get_objects_node()
        myvar = root.get_child(["0:Objects", "2:MyObject", "2:MyVariable"])
        variableObj = value(client, myvar)
        print(variableObj.MyVariable)

        while True:
            variableObj.write_to_sql()
            time.sleep(10)

    finally:
        client.disconnect()

Now every 10 seconds your variable class will have it's sql function called. That function just needs to do INSERT self.MyVariable INTO self.table_name

The SubHandler call will still happen on the network thread, so even though your main thread is sleeping variableObj's MyVariable attribute will be kept up to date on every data change. The downside of this is you get timed updates, instead of datachange updates. That was why I was going in the direction of using __setattr__. A workaround for this would be to just see if MyVariable has changed since the last time write_to_sql() was called. If the data hasn't changed, don't do the insert. I hope you get the idea.

[tdesroches]

That would work too actually much easier than threading lol

[tdesroches]

If the SQL db is on localhost is it still a bad idea to perform the query in the handler?

[zerox1212]

SQL interactions are normally blocking, so you wouldn't want to do it according to @oroulet .

[zerox1212]

Although in your original code you are already doing it, so it will probably work.

[zerox1212]

BTW, you could read this to make the code safe from timeouts on the network thread. https://stackoverflow.com/questions/13763685/how-to-run-a-function-called-in-a-thread-outside-that-thread-in-python

And you still don't need to make a separate thread, just use the main thread to process the queue as described in that link.

[zerox1212]

I also forgot to mention. It looks like your code is just doing historizing to SQL. The python OPC UA server has built in history using memory or SQLite. So you could probably just do historizing of node values on the server side. Then your client only has to query the OPC UA servers node history data. The other advantage to this is that almost any OPC UA Client or historian will also be able to view the historical data.

[tdesroches]

Thanks zerox I'll take a look at both of those. Appreciate the help :)

[tdesroches]

Would the solution we worked out today be worth submitting to the examples documentation?

[oroulet]

Like like there is a bunch of people wanting to sync UA nodes with s python object. Maybe we should write some code for it. Isn't there an example already?

[zerox1212]

The example I made that is already in the repository seems to only work with a UA Server, because i'm doing browse_name lookup in the handler. Maybe we should update it so it can be used more universally.

[oroulet]

@zerox1212 yes remove the network call and cache the node id. The nodeid is sent back at every data change callback