Extend Python OPC UA to support PubSub Standard for Cloud Protocols

[zerox1212]

Can we look at adding this functionality natively to support the various cloud platforms that are becoming more and more prevalent? Also I'm on a project that needs to send data to Azure. ;)

https://opcfoundation.org/news/opc-foundation-news/opc-foundation-announces-support-of-publish-subscribe-for-opc-ua/

My other project https://github.com/zeroentity/opcua-2-azure is kind of a workaround for this standard. It would be nice if someone on here is familiar with AMQP protocol to help implement.

I think even the https://github.com/OPCFoundation implementation doesn't have this functionality yet (looks like they made a hack similar to my project here https://github.com/OPCFoundation/UA-.UWP-Universal-Windows-Platform/tree/master/SampleApplications/Samples/Publisher). Would be nice to beat them. :) They also don't support python.

I think it would not be that difficult, just parse subscribed data into JSON and send out to AMQP endpoint. This could be loaded as a module, similar to how history is designed.

[oroulet]

Interesting but we need spec or sample implementation. There are many ways to implement opcua over amqp. If you find spec we can have a look

[zerox1212]

I have asked OPC Foundation for information, however I think the specification is not actually released yet.

A basic implementation will not be that difficult, but reporting every data change to Azure as it's own message will result in a LOT of AMQP messages. That would be very expensive for a per-use billing such as Azure. Hopefully the spec will try to collect a list of data changes and send a single AMQP message.

Anyways, I will try to design my side project to have a flexible module for converting OPC UA objects to JSON. Maybe that way we can reuse it.

[zerox1212]

I found some more information. http://www.automation.com/automation-news/industry/opc-ua-is-enhanced-for-publish-subscribe-pubsub

And an open source project which appears to be implementing a full featured pub sub library. https://github.com/mpostol/OPC-UA-OOI/blob/master/Networking/SemanticData/README.md

This OPC-UA-OOI looks like a very advanced implementation. I don't understand hardly any of the diagrams in the readme...

[oroulet]

So we need spec part 14. Looks like the XML is available there https://github.com/OPCFoundation/UA-Nodeset/tree/master/Schema So we can already generate the necessary structures

[zerox1212]

Quite interesting that the XML is already there.

I got a message from an OPC Foundation person and they said the pub sub spec is only available to the Working Groups. The public release of the spec won't be until the end of 2017.

It would be interesting if you can generate from the XML and see what shows up.

[oroulet]

you can try yourself. modify a file called something like generate_protocol.py under schemas

On Fri, 10 Jun 2016 at 17:33 Andrew notifications@github.com wrote:

Quite interesting that the XML is already there.

I got a message from a OPC Foundation person and they said the pub sub spec is only available to the Working Groups. The public release of the spec won't be until the end of 2017.

It would be interesting if you can generate from the XML and see what shows up.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/FreeOpcUa/python-opcua/issues/205#issuecomment-225215711, or mute the thread https://github.com/notifications/unsubscribe/ACcfzjWdLK27r2HprSQkJ9WyEG_0SGvxks5qKYOogaJpZM4IxRbY .

[zerox1212]

I have never tried to generate the code from the XML. I will try it if I get time.

[zerox1212]

I tried to generate the code with part14, but nothing shows up. Do you have some hints about how this code generator works? There are almost no doc strings.

[oroulet]

the pub sub protocol will not only suport amqp but also UDP. This is good. A drop of info there https://opcfoundation.org/opc-connect/2016/06/toolkit-case-study-and-pubsub-white-paper/

[gillerman]

The XML from the OPCF has been updated: https://github.com/OPCFoundation/UA-Nodeset/tree/master/Schema

Did anyone figure out how to gen the code?

I am happy to help test, but I don't have the skills to write the code.

[oroulet]

Generating the classes is easy. Will add it for fun. But we need more than the xml to support the protocol: a spec or sample client/server to reverse engineer

[gillerman]

Can you develop against the Microsoft .Net stack? https://github.com/OPCFoundation/UA-.NETStandardLibrary

https://blogs.microsoft.com/iot/2016/06/23/microsoft-introduces-new-open-source-cross-platform-opc-ua-support-for-the-industrial-internet-of-things/

[zerox1212]

We have looked at it before briefly. It would be an undertaking to implement.

What exactly do you want to do? Do you need to develop in Python? Why not just use the .NET stack which you posted?

If you just want to publish some OPC UA data to Azure IoT Hub that is actually easy to do because of Microsoft's IoT SDK. I have done it several times.

[oroulet]

Does the .net stack support publish subscribe?

[gillerman]

Yes. The subscriber should work completely. I think they are finishing up the publisher. I may be wrong, but all this will supposedly be done by Microsoft by the end of September. Note there are currently two .Net stacks...

[oroulet]

May I ask what you want to do with the python stack? Can you help?

[gillerman]

Develop OPC UA publishers and subscribers that can communicate over AMQP and eventually, TSN (UDP). Ideally, it would be good if these components were integrated with both the UA Server and UA client so that a UA client and do pub/sub and a UA Server can do pub/sub. I can help test. I am not a good enough programmer to create an SDK. For example, I don't know how to wrap a .Net stack with python to create an SDK. Pure Python would be easier for me.

[zerox1212]

Everyone here would vote to do a pure python implementation, but it would be an undertaking. I looked into it and I don't think a python AMQP implementation exists. Same for MQTT. Probably only C wrappers.

I looked through the .NET repository quickly and I didn't see much for documentation. Seems they have some kind of publisher, but I'm not sure how it's working. The OPC Foundation spec is also not even released yet, it's only a draft.

[gillerman]

We need to support AMQP 1.0. https://qpid.apache.org/releases/qpid-proton-0.13.1/proton/python/api/index.html

[zerox1212]

We would need to check that library. It appears to be a wrapper for a c implementation. If it's not platform specific we could use it.

It's time for you to start learning Python and make a PR. :) If @oroulet generates the python code for you, all you need to do is read the OPC foundation spec and reverse engineer the .NET sub pub implementation.

[oroulet]

Having a backend depending on a c lib is not a big issue as long as the entire lib does not depend on it. But personally I am more interested in the UDP backend

[zerox1212]

Unfortunately I'm sure Microsoft is more interested in AMQP so that they can push Azure to industrial companies. Having worked in the sector for a while now, I can see why. Getting industrial data to Azure is not very straight forward (not to mention the amount of data a factory generates is expensive to cloudify).

Did you see anything that looks like UDP in the .NET stack?

[oroulet]

https://github.com/FreeOpcUa/python-opcua/pull/289

[gillerman]

There are many large companies very interested in running OPC UA pub/sub UDP over TSN. My guess is that embedded Windows IoT will support UDP after they get AMQP going.

[oroulet]

Anyway UDP or amqp is about the same for us probably 90% common code. We just have to remember to add possibility for different backends

.

[zerox1212]

Thanks for generating the code. Now we just need to find time to take a look at the spec/reverse engineer .NET and plan an implementation.

[ghost]

It looks like the opcfoundation released a specification which supports Pub/Sub. https://opcfoundation.org/developer-tools/specifications-unified-architecture/part-14-pubsub/

Will you implement Pub/Sub over AMQP?

[zerox1212]

Part 14 was already converted to python objects here https://github.com/FreeOpcUa/python-opcua/pull/289 . However at this time no one is implementing Pub/Sub. I have a feeling if someone got it started others would contribute, but getting started is the hardest part.

[cercata]

What's the status of this ?

[zerox1212]

Same as March. The project I was on died so I no longer have a use case. I can't dedicate any time to implementation. If you want to prototype something for us to review people might start to work on it.