Some extension ideas :)

[joeblew99]

thanks for this example - really easy to follow along.

I am wanting to put a Message queue within this architecture and wanted to tun it by you and well the whole internet :) I am still trying to work out the best way, without introducing too much boiler plate into it.

Why do this ? Because then a GRPC Service can also call other GRPC services as and when they need to. This allows building larger systems of Microservices where the Choreography is emergent. Bottom up rather than top down. Essentially its the reflux pattern being used for Microservices. Nothing ground breaking but an important principle i feel.

Here is what i intend to do: The API code where the REST / JSON layer is exposed would put JSON messages onto a Message queue topic. Then the GRPC Service would listen for them on the right topic. Nothing really to it.

a Coupe of design choices thogh :)

GRPC-gateway Why not just use the GRPC-gateway to expose the GRPC services as Resftul JSON services ? The pro is that you get code gen for free. Your API code is much less boiler plate and so much less brittle. I think also you get websockets for free based on the latest code there.

NATS or NSQ ? Apart from HA style semantics, the thing that is also important is how easily it can handle websocket. The reason being because then a Microservice Or the APi layer can also push to the client (s). Dont knwo about you, but for me, its an important this. I like to design (and think about the client) as just being another service behind a message queue. SO the centre of everything is your API layer where everything comes in and out of. There are not many examples of using NQS or NATS with web sockets from what i can find.

Would be cool if you have any thoughts on this. I am happy to make a PR for this and contrib it back to this repo too. There are not enough examples of this around i feel.

[harlow]

@joeblew99 thanks for the note - totally agree that queues are an important part of microservice architecture. NSQ feels like a pretty solid choice here, and from my limited experience with it, seems like a great choice for applications that need to communicate in an async fashion.

Would be cool if you have any thoughts on this. I am happy to make a PR for this and contrib it back to this repo too. There are not enough examples of this around i feel.

Sounds good. Any ideas on functionality?

[joeblew99]

thanks for the response :)

Can you comment on why your not using grpc-gateway. It would get right a some of your boiler plate. I hate typing :)

• Easy to have a go at it too.
• You get a swagger API fro free too.
• Also grpc-web is ongong private repo, and so it will be easy to swap out when its ready later.

Now about the idea of putting in a Message queue layer and functionality:

• With a MQ, then all the good things of CQRS are then thee for the taking.
• Providence / Audit trail from the event store logs
• Ability to relay logs to simulate a bug or crash
• No need for DB migration scripts, because the event store is the full history. Its similar to how the blockchain works. When you change anything in the system just replay the logs back into the system and lets the read and write databases fill up. Its weir to get used to, but its there and makes doing advanced architectures such as machine learning possible for example.

Lbs that work well with your setup: https://github.com/andrewwebber/cqrs

• Very well documented
• Clear

https://github.com/nvcnvn/nsq_to_couchbase

• Needed to store logs for every
• Couchbase is all golang and makes it easy to go HA.
• Your events will have Full facet search. Huge
• I woudl also pipe all application logs and metrics into it for pushing off to Prometheus later. Again the facet search is huge here because you get to have amazing tooling for search for free.
• I would even put tracing into it. But not sure yet.

Coach DB is a controversial choice for Message queue backing i know. Couch has come along really well now and is 100% golang with offline support and full facet and text search integrated. Its compelling. Its clear that Microservices shoudl use what they want, but i need to pick one DB as a basis for the infrastructure layer of the control plane. Hope you get me.

I dont think its a huge thing to adapt what you have now to start to use some of this.