Architecture related questions

[Andrey9kin]

Hi!

We are working on very similar to remrem implementation of Eiffel protocol and ecosystem around it. I didn't know about remrem when we started so now I'm trying to see how much we can reuse and then contribute to remrem instead of producing duplicates.

1. One obvious candidate is remrem-semantics. I already posted an issue to discuss possibility to use protobuf instead of manually coding more or less empty classes https://github.com/Ericsson/eiffel-remrem-semantics/issues/20 If we can have a discussion around this and agree on the way forward, then we will either contribute what we did using protobuf or if there are good arguments against that then we will drop it and continue working with remrem-semantics.
2. Another point of interest is a shared library to encapsulate broker details. I guess it will be a shared library between microservices to encapsulate logic around RabbitMQ - establishing a connection, handling connection breakdowns, configuration and so on. This library would be the interest for us and we would love to contribute development efforts.

Just to give you an overview of our current best thinking:

Here are components as we seem them (tracey is a project name, derived from word trace)

Brokers (currently we are looking only into rabbitmq but we would like to leave a room for the other implementations and allowing architecture to support different brokers) tracey-broker-rabbitmq tracey-broker-sqs ...

tracey-core - a core component that glues together broker and input components

Protocols (currently only Eiffel but again with possibility to add more protocols) tracey-protocol-eiffel ...

CLI: tracey-eiffel-rabbitmq-cli - CLI tool to send Eiffel messages using RabbitMQ (this is how we think to simplify architecture - have vendor specific builds) tracey-eiffel-sqs-cli - CLI tool to send Eiffel messages usign Amazon SQS ....

Jenkins: tracey-eiffel-rabbitmq-pipeline-sender-plugin - you got the point tracey-eiffel-rabbitmq-trigger-plugin ....

Looking forward to your comments, Andrey

[d-stahl-ericsson]

We're trying to go the microservices route, as we believe it is an architectural style that suits the ecosystem we're building very well. To that effect, Remrem is intended to be the abstraction that encapsulates broker details and serialization. So rather than having multiple components integrate a common library, they instead make requests to a shared service.

[Andrey9kin]

@d-stahl-ericsson yes, I see that. What I mean is that publish and subscribe services will interact with the RabbitMQ in a very similar way when it comes to connection, exception, configuration handling, and it probably makes sense to decouple this part to a shared library. Then we could wrap this library, semantics into a command line and provide complimentary CLI tool that could be used while microservices are under development and for testing purposes. Going forward we can replace direct RabbitMQ calls with REST HTTP API calls to microservices

[d-stahl-ericsson]

I would much rather see Remrem being "CLI wrappable" for such use cases, rather than breaking out a shared library. I.e. local spin-up of the service, curl localhost and take it down.

[Andrey9kin]

@d-stahl-ericsson I understand and agree. But we should also think about end user convenience. For instance, extracting all necessary info from Git commit, forming request and sending it requires certain automation so it is more than running curl. That is why I see a need for a CLI client. For a time being this client will send messages using RabbitMQ directly and then will switch to HTTP calls when there is a possibility to do that, i.e. when microservice is mature enough.

[d-stahl-ericsson]

Of course there's more to it to formulate the message - I never intended to imply anything else. My point is that invoking the service (remote or locally spun up) should be as simple as invoking a library function. I've asked @danielyinanc to provide an example of just that, and he has promised to do so. It is my understanding that Remrem is at or very near that level of maturity already.

[Andrey9kin]

@d-stahl-ericsson @danielyinanc I looked a bit more into remrem implementation and so far it is unclear for me what exactly problem we are trying to solve when using microservices instead of components. As far as I can see client still have to know about broker specifics - broker vendor (configuration is RabbitMQ centric), broker address, broker specific routing info, i.e. exactly the same info that you would need to know when operating RabbitMQ client. So what is the difference comparing to calling RabbitMQ client? With this we are adding one more level of complexity, what is the gain? Please help me to understand your thinking. If the main goal is modularity and decoupling then we can achieve the same level of modularity by using components (read set of shared libraries) with well-defined interfaces.

[d-stahl-ericsson]

Remrem should abstract away broker specifics. I must confess my ignorance with regards to the details here, but what you're describing sounds like a problem to me. That kind of configuration information is something I'd rather see done once when launching the service - not something every client does for every request.

[Andrey9kin]

@d-stahl-ericsson sorry I'm a bit incorrect here - messed up what remrem should know with what client supposed to know. So remrem should obviously know about rabbit mq server and all that details, client shouldn't. What client should know and provide is rabbit mq specific routing info. It would be interesting to see how we can abstract that. If we manage to do so then yes client won't know any broker vendor specifics. Thanks for pointing that out :) I will close this thread for now. Let's discuss it more during Eiffel summit