search

tcalmant / ipopo

iPOPO: a Service-Oriented Component Model for Python
https://ipopo.readthedocs.io/
Apache License 2.0
69 stars 28 forks source link

Remote services vs Cohorte Herald #81

Closed enferre closed 6 years ago

enferre commented 7 years ago

This project is awsome and I'm really thinking to use it in my projects. I hope you are going to keep developing and maintaining it! One question: how iPOPO remote services are related with Cohorte Herald project? Do you suggest to exploit Herald or the transport/discovery provided with iPOPO

tcalmant commented 7 years ago

Thanks :) I've less time to work on iPOPO that I'd like, but don't worry, I'll continue to maintain it.

About Cohorte Herald vs. Pelix Remote Services: both have different purpose:

So, in short: if you only want to call methods from services of other iPOPO frameworks on a common network, just use the JSON-RPC provider of the Pelix Remote Services. If you also want a communication layer and to ease communications with a Java framework: use Cohorte Herald.

Note that Cohorte Herald provides an XMPP-based discovery and RPC transport, which is very useful when needing to connect frameworks on different networks or even geographically distant.

A bit of history, to have more context: my PhD thesis led to the development Cohorte Framework, which allows to develop component-based applications based on Java & Python components and with some automatic restart options. The components are instantiated in different processes on different machines. As it was based on Java/OSGi/iPOJO from the beginning, I started iPOPO to have the equivalent in Python and to ease the overall development. At first, components of different languages couldn't talk to each other: Java components components were based on Apache CXF DOSGi then Eclipse ECF, while I developed the Pelix Remote Services for iPOPO (inspired from the ECF architecture). Then I ported the Pelix Remote Service API to Java, to allow both languages to communicate through Jabsorb-RPC (JSON-RPC with class hints).

Cohorte Herald was created a year after that to allow inter-process discovery and messaging. As a message can contain anything, I developed a version of JSON-RPC and Jabsorb-RPC over Herald messages. As a result, Cohorte Herald is now a discovery and a transport provider for Pelix/Cohorte Remote Services.

scottslewis commented 7 years ago

Hi Thomas and enferre,

Although it's been almost a year since I've had a chance to work on it, I would like to continue working on iPopo in remote services support. Specifically, I would like to introduce an impl of RemoteServiceAdmin (RSA) that was standardized in OSGi 4.3 spec some time ago.

I'm the lead developer on the ECF project https://wiki.eclipse.org/ECF, we have a spec-compliant impl of OSGi Remote Service Admin 1.1, and I would like to contribute an iPopo-based impl of RSA to iPopo. The value of this would be:

1) Standardization of remote services meta-data that allows consistent discovery and usage across Java (OSGi) and Python (iPopo);

2) The support for a runtime-pluggable distribution and discovery, allowing the use of variety of existing and/or new transports and/or serialization formats (e.g. Jabsorb, Cohorte, REST-others, mqtt, xml/json, protobuf, JMS/other messaging, custom/private transports, etc). From the service creator and consumer perspective, this allows the creation and use of transport independent services.

3) A standardized remote service management API, to allow remote services to be coordinated/managed.

4) Use of injection frameworks of choice on both Java side (e.g. ipojo, ds, etc) and python side (ipopo) based around service registry and remote services.

I've been working with folks at Intel Internet of Things, using remote services in an OSGi framework to implement Java<->Python remote services using Py4j and an 'osgiservicebridge':

https://github.com/ECF/Py4j-RemoteServicesProvider

I've already made this into a distribution provider (also called a remote services provider) for the java side and I was going to use it as one of the RSA providers on the ipopo side (in addition to Jabsorb/Cohorte...which is already been turned into a distribution provider in this project https://github.com/scottslewis/cohorte-remote-services.

We are now using java remote services implemented in python and we are contemplating use ipopo + RSA to go the other way and get pelix dynamics, ipopo injection, a consistent way to do transport-independent remote services and RSA, etc.

I also have some early code implementing RSA on pelix/ipopo that I intend to dust off and use to complete this contribution.

On Sat, Jul 29, 2017 at 3:05 AM, Thomas Calmant notifications@github.com wrote:

Thanks :) I've less time to work on iPOPO that I'd like, but don't worry, I'll continue to maintain it.

About Cohorte Herald vs. Pelix Remote Services: both have different purpose:

  • Pelix Remote Services can be used to let Pelix/iPOPO frameworks communicate through RPC, in point-to-point connections. By using the Jasborb-RPC provider (included), you can also communicate with Java/OSGi services if the Java frameworks runs the Cohorte Remote Services https://github.com/isandlaTech/cohorte-remote-services.
  • On the other hand, Cohorte Herald is primarily a message broker: messages can be send to a peer or a group of peer. It was created to let peers discover each other and communicate with a total abstraction of the protocols underneath. It provides an RPC layer based on messages that can be used by Pelix and Cohorte Remote Services.

So, in short: if you only want to call methods from services of other iPOPO frameworks on a common network, just use the JSON-RPC provider of the Pelix Remote Services. If you also want a communication layer and to ease communications with a Java framework: use Cohorte Herald.

Note that Cohorte Herald provides an XMPP-based discovery and RPC transport, which is very useful when needing to connect frameworks on different networks or even geographically distant.

A bit of history, to have more context: my PhD thesis led to the development Cohorte Framework, which allows to develop component-based applications based on Java & Python components and with some automatic restart options. The components are instantiated in different processes on different machines. As it was based on Java/OSGi/iPOJO from the beginning, I started iPOPO to have the equivalent in Python and to ease the overall development. At first, components of different languages couldn't talk to each other: Java components components were based on Apache CXF DOSGi then Eclipse ECF, while I developed the Pelix Remote Services for iPOPO (inspired from the ECF architecture). Then I ported the Pelix Remote Service API to Java https://github.com/isandlaTech/cohorte-remote-services, to allow both languages to communicate through Jabsorb-RPC (JSON-RPC with class hints).

Cohorte Herald was created a year after that to allow inter-process discovery and messaging. As a message can contain anything, I developed a version of JSON-RPC and Jabsorb-RPC over Herald messages. As a result, Cohorte Herald is now a discovery and a transport provider for Pelix/Cohorte Remote Services.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/tcalmant/ipopo/issues/81#issuecomment-318818534, or mute the thread https://github.com/notifications/unsubscribe-auth/AAJw9HOJkmUhKFUDWDK8wHtJRFXRL2ZOks5sSwQEgaJpZM4OnRen .

enferre commented 7 years ago

Dear Scott and Thomas,

thank you for having shared these information and links. I'm going to carefully have a look on them. The sinergy between your projects look promising and I am looking forward to see progresses! For your knowledge, at the moment I'm looking at Pelix/iPOPO as best candidate for supporting the implementation of the IoT resource adaptation layer under development within EU-funded project MONICA: http://www.monica-project.eu/ . Sorry for the self-advertising :-)

BR, Enrico

tcalmant commented 7 years ago

Hi Scott and Enrico,

That's great news ! Thanks Scott for working on the RSA, I think it will be a major feature for iPOPO 👍

Also, it would be really interesting to have feedback on the usage (or non-usage) of iPOPO in your projects :)

Best Regards, Thomas

tcalmant commented 6 years ago

FYI, the RSA implementation has been added to iPOPO. The documentation is available here: https://ipopo.readthedocs.io/en/latest/refcards/rsa.html

enferre commented 6 years ago

Great! I'll have a look on it. Thank you!

BR, Enrico

Il dom 19 ago 2018, 15:25 Thomas Calmant notifications@github.com ha scritto:

FYI, the RSA implementation has been added to iPOPO. The documentation is available here: https://ipopo.readthedocs.io/en/latest/refcards/rsa.html

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/tcalmant/ipopo/issues/81#issuecomment-414127696, or mute the thread https://github.com/notifications/unsubscribe-auth/AJjvAWGyLFArMb2iUx-ZgfIfhfNx9Ebjks5uSWc5gaJpZM4OnRen .

scottslewis commented 6 years ago

Hi enferre

On Fri, Jul 28, 2017, 11:26 PM enferre notifications@github.com wrote:

This project is awsome and I'm really thinking to use it in my projects. I hope you are going to keep developing and maintaining it! One question: how iPOPO remote services are related with Cohorte Herald project? Do you suggest to exploit Herald or the transport/discovery provided with iPOPO

The RSA impl supports multiple distribution (transport) and discovery providers.

We intend to add eg cohorte but have not had chance to do so yet.

The good news is that the dist and discovery apis are there with examples and api doc. so we expect to come quickly...especially with contributions and/or help testing. Part of moving to rsa was formalizing this multi-provider arch some.

I welcome new issues to support/use other providers. Will get to them as fast as possible...and help others as needed.

—

You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/tcalmant/ipopo/issues/81, or mute the thread https://github.com/notifications/unsubscribe-auth/AAJw9AQ_qUPwI_gVuwN8XDAD_iAH8x7Tks5sStCFgaJpZM4OnRen .