CQRS.NET, AKKA.NET, MassTransit and KAFKA.NET

[eaba]

I Love these projects and would love to use them in my project. I trying hard to see where all will compliment and fit with each other.

How do I get a guide from you? Or Can I get a guide from you?

Kinds Regards!

[cdmdotnet]

What benefit of each project are you wanting to utilise? What components or features of a CQRS platform are you wanting to be implemented by each different technology?

We'll need to know that in order to guide you.

[eaba]

I would love to separate Requests and Queries(CQRS) and Maintain State in each Layers(AKKA.NET) with Event Store(KAFKA) using MassTransit as the underlining Transport Bus! I am basically thinking in line with "Software Pipelines" without bottlenecks and high(VERY HIGH) TRANSACTION PER SECONDS

[cdmdotnet]

Ok. We're still working on detailed guides and documentation but in your case you'd want to look at the SqlEventStore class as an example of how to write an EventStore using KAFKA. Otherwise there is the MongoDbEventStore class. Basically there's two Get methods and a PersistEvent method to override. The rest is just stuff relevant to settings and any specifics of KAFKA. For MassTransit and networking you'd be best to refer to the Cqrs.Azure.ServiceBus project. In that you are basically implementing 6 interfaces. IPublishAndWaitCommandPublisher. ICommandReceiver. ICommandHandlerRegister. IEventPublisher. IEventReceiver. IEventHandlerRegistrar.

Again everything else in that project is settings and Azure related stuff.

As for Akka.net we already have a module for that to ensure concurrency allowing you to use other persistence and networking technology... which sounds like your position. There's a test class that shows 4 test commands, events and a mix of Akka.net and non Akka.net handlers all working together.

If you make a fork and make a pull request we'll review what you are doing and help you out developing it... we could even merge it to provide better assistance and support going forward.

Is that the type of direction you are thinking of or have we misunderstood what you are aiming to do?

[eaba]

I also want to create dynamic pipeline using Content-Based routing

[cdmdotnet]

After that piecing it all together will mostly be a case of changing your IOC configuration... CQRS.NET is big on interfaces so you can swap in and out things as you want to.

[cdmdotnet]

Currently Content-based routing is pretty simple. The Azure ServiceBus for example allows you to define filter rules which dictates if a specific message (so command or event) will be sent to a specific node/server etc. It's rather basic and primitive right now but serves a similar purpose that can be controlled either from a CQRS.NET application or externally from Azure itself. That might be helpful to see what we've done there... but it might also be too primitive depending on the specifics of your requirements. We're at early stages of looking at KAFKA so we can't speak too much at the moment about it while we're still learning it ourselves.

[eaba]

Is that the type of direction you are thinking of or have we misunderstood what you are aiming to do?

it is on track with my aim and I will update you as I progress. Thanks

[cdmdotnet]

Just as an FYI, there's some conversation on the pros/cons/things to think about when using KAFKA as an event store on stackoverflow

[eaba]

Thanks @cdmdotnet

[eaba]

The book: Big Data SMACK: A Guide to Apache Spark, Mesos, Akka, Cassandra, and Kafka gave me some idea also