Publish events to Spring Cloud Stream channels

[mehdichitforoosh]

Hi. Can Axon framework provide publish events directly to Spring cloud stream? Or we should use spring-amqp in event-driven microservices based on Spring cloud? Thanx.

[smcvb]

This extension is currently mainly focused on using Spring Cloud discovery to route commands from one Axon instance to another.

I'd assume Spring Cloud Stream would be a workable fit to route events from one instance to another, sure, but that's not what this extension is currently focused on.

A pull request to provide this functionality would definitely be valuable of course! If however you're hard pressed to provide that, then I'd like to recommend you to use Axon Server to route any type of message in your system (note that Axon Server has a free edition). Biggest benefit of picking Axon Server is that you have to provide very little configuration at all.

If this is not a workable fit for you, you can indeed use the Axon AMQP or Axon Kafka (still in beta) extensions.

Ow and lastly, @mehdichitforoosh, would you mind sharing whether you'll provide a PR to implement Spring Cloud Stream support for message?

[mehdichitforoosh]

@smcvb Hi. Yes I'm interested in doing this but need help in reading the codes and integrate Axon API with Spring Cloud Stream.

[mehdichitforoosh]

@smcvb It seems we should start with implementing EventBus interface in org.axonframework.eventhandling package with Spring Cloud Stream API in publish methods and auto configure the SpringCloudStreamEventBus for avoiding creation of SimpleEventBus in AxonAutoConfiguration. Do you agree?

[smcvb]

An EventBus would very likely be a step to far in this scenario.

Firstly, you need to deduce whether the Spring Cloud Stream API is a push or pull mechanism. Based on this, you can figure out whether this approach could be an event source you subscribe on (pushing message) or an event source you can stream from (pulling messages).

Both solution have a dedicated interface in Axon, respectively the SubscribableMessageSource and the StreamableMessageSource. An EventBus for example only implements theSubscribableMessageSource, whilst an EventStore also is a StreamableMessageSource.

To see actual worthwhile code samples for both usages, I'd suggest you have a look at the Axon AMQP Extension for the subscribing solution and Axon Kafka Extension for the streaming solution.

[mehdichitforoosh]

Spring Cloud Stream API is push and also a pull mechanism(Source,Sink and Process interfaces).It has SubscribableChannel for input messages and a MessageChannelfor output messages from Spring Messaging package.It seems we should use both of EventStore and EventBus.I am currently reading source codes from Axon Framework and Spring Cloud Stream to understand relationships between objects. I think solutions from Axon AMQP Extension can help.
Soon, I'll share with you what I've find out for implementing classes. Thanx.

[smcvb]

Good stuff @mehdichitforoosh! However, I'd like to suggest to focus on the SubscribableMessageSource and StreamableMessageSource interfaces in the framework than. Both message sources are the required inputs for respectivelky providing events to a SubscribingEventProcessor and a TrackingEventProcessor, so staying closer to that API is beneficial I think.

[mehdichitforoosh]

Hi. @smcvb @abuijze There are classes with names InboundEventMessageChannelAdapter and OutboundEventMessageChannelAdapter in Axon Spring module based on Spring Messaging MessageChannel and MessageHandler interfaces. Spring Cloud Stream is based on Spring Messaging MessageChannel interface,too. If we configure Spring Cloud Stream and inject Output channel (MessageChannel) object created by Stream to OutboundEventMessageChannelAdapter object for sending messages and subscribe InboundEventMessageChannelAdapter object as a MessageHandler to SubscribableChannel object from Stream.It seems to work without developing more stuff.But I have not tested yet. But for PollableMessageSource interface that have been added in Spring Cloud Stream 2.0,I think some classes are needed.
Do you agree?

[mehdichitforoosh]

Hi. I don't have permission to push the code changes to axon-springcloud-4.1.x branch. I mistakenly created a new pull request.This is the first time i want to contribute in a github project.:-) Please check these examples for integrating Axon Framework with Spring Cloud Stream and Spring Integration. Integrating Axon with Spring Cloud Stream It works for me.Of course this is an initial version. @smcvb @abuijze

[smcvb]

Thanks for your insights on this @mehdichitforoosh, and amazing to hear that Axon is the first Open Source project you are contributing too! Definitely an honor to us.

What you should have done to issue a pull request, was to fork this project and create a branch dedicated for this implementation there. From there, you should be able to provide your forked branch as a pull request to the framework.

[satvista13]

@mehdichitforoosh Thanks for the info. I am able to successfully publish axon messages from the Command side to the spring channel, serialize it with AVRO and send it to Kafka. But Integrating the Spring's Subscribable Channel in the Query side still remains a concern if we go for AVRO deserialization. Please share your thoughts. @smcvb This seems to be a requirement on many Event Driven Platforms that is to have a very good CQRS+ES APIs. Axon seems to be perfect but it is quite challenging to integrate it with our Stream Processing pipeline. Any updates from Axon on this would be great!

[smcvb]

Let me provide some background here @satvista13.

Note that I have tried to be clear that the Spring Cloud Extension is meant for Command Message Routing. The PR which has been provided by @mehdichitforoosh is dealing with Event Message Routing.

From the stance of Axon Framework, there are three very distinct types of messages you deal with, being Commands, Events and Queries. All three have very different routing needs too:

• Commands are routed to a single handler
• Events are distributed to anyone who's interested
• Queries require load-balanced routing

This is also emphasized by @abuijze in this video.

That said, to cover the issue of making your own integrations, at AxonIQ we are putting a lot of effort in to creating Axon Server. Axon Server is a dedicated solution to routing Commands, Events and Queries taking the routing needs into account (as well as being a dedicated event store).

This seems to be a requirement on many Event Driven Platforms that is to have a very good CQRS+ES APIs.

This point is exactly why we have created Axon Server. As such I am highly recommending you take a look in to that project. It greatly alleviates the non-functional requirements of connecting components (or thinking in problems like AVRO as a message contract and Kafka as a bus) and let's you as a developer focus on providing business functionality first.

Due to this emphasize, we are somewhat thinly spread when it comes to the extensions. As I have pointed out to @mehdichitforoosh, his Pull Request process should be dealt with a little differently. I will first wait for his participation on this prior to disclosing any time lines on when a Spring Cloud Streams Extension dedicated for Event Message Routing is completely resolved.

[satvista13]

Thanks Steven. Appreciate your insight and help Just FYI, We are building a new platform of event driven microservices to replace our .NET applications and data processing pipelines to replace our Mainframe backend. For Stream processing Spring Cloud Stream apps on top of Kafka has been the decided approach and we working towards finding ways to integrate our API Microservices ( CQRS + ES is required) with our data pipeline. Axon seems to be a perfect fit ( we did evaluate eventuate too) and we'll be putting it in place.

i'll look into Axon server and see if it fits our needs. We are planning to deploy our apps in Azure Open shift. Quick Question How Command Bus is implemented in Axon Server. Is it a HTTP Proxy or an MQ type like Kafka/Rabbit? Thanks, Sathish

On Thu, Aug 1, 2019 at 10:20 AM Steven van Beelen notifications@github.com wrote:

Let me provide some background here @satvista13 https://github.com/satvista13.

Note that I have tried to be clear that the Spring Cloud Extension is meant for Command Message Routing. The PR which has been provided by @mehdichitforoosh https://github.com/mehdichitforoosh is dealing with Event Message Routing.

From the stance of Axon Framework, there are three very distinct types of messages you deal with, being Commands, Events and Queries. All three have very different routing needs too:

• Commands are routed to a single handler
• Events are distributed to anyone who's interested
• Queries require load-balanced routing

This is also emphasized by @abuijze https://github.com/abuijze in this https://www.youtube.com/watch?v=d1nsaPpRSMA&t=260s video.

That said, to cover the issue of making your own integrations, at AxonIQ we are putting a lot of effort in to creating Axon Server https://axoniq.io/product-overview/axon-server. Axon Server is a dedicated solution to routing Commands, Events and Queries taking the routing needs into account (as well as being a dedicated event store).

This seems to be a requirement on many Event Driven Platforms that is to have a very good CQRS+ES APIs.

This point is exactly why we have created Axon Server. As such I am highly recommending you take a look in to that project. It greatly alleviates the non-functional requirements of connecting components (or thinking in problems like AVRO as a message contract and Kafka as a bus) and let's you as a developer focus on providing business functionality first.

Due to this emphasize, we are somewhat thinly spread when it comes to the extensions. As I have pointed out to @mehdichitforoosh https://github.com/mehdichitforoosh, his Pull Request process should be dealt with a little differently. I will first wait for his participation on this prior to disclosing any time lines on when a Spring Cloud Streams Extension dedicated for Event Message Routing is completely resolved.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/AxonFramework/extension-springcloud/issues/2?email_source=notifications&email_token=ALG5CVHIE4KPIRTDEFPDYRTQCLWLJA5CNFSM4GZBMTJKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3KYB7Y#issuecomment-517308671, or mute the thread https://github.com/notifications/unsubscribe-auth/ALG5CVBYAZ36LP5WW2BKEATQCLWLJANCNFSM4GZBMTJA .

[smcvb]

@satvista13 it would be better suited to discuss a question like this on the User Group or to contact us at AxonIQ regarding this. That way we can maintain focus in this issue on the main request, which is to to provide an Event Message Routing solution to Axon using Spring Cloud Streams.

[mehdichitforoosh]

Hi @smcvb . Sorry for the delay in replying. Yes,I want to complete this pull request. Please guide me what changes I need to make.

[smcvb]

No worries @mehdichitforoosh. Let's focus on pull request #5 to discuss this further instead of distributing the discussion over the issue and the PR.