Add asynchronous message support, with Kafka

[jdubois]

One important feature we currently don't provide is asynchronous messaging. I'm not listing here why it's a good thing :-)

There are a number of implementations to achieve this: with ActiveMQ, RabbitMQ, Kafka, Redis, etc... And of course we need a good API for this.

I've already got something working with Kafka, but I'm using Spring Kafka directly: I'm not satisfied with it, but I know how to do it, and I know it works :-)

My current idea is to:

• Add Spring Cloud Stream support as it's a layer on top of RabbitMQ, Kafka, Redis, etc... So we would not be locked on Kafka. I haven't used it yet, and I'm not sure of the limitations -> if I'm not happy with it, I'll go back to "pure Kafka"
• Use Kafka as an implementation: mostly because I already have it working (including Docker images, etc), but that's also the server for which I see the biggest demand/hype at the moment.

Then that would add a new question on the generator, so my idea is to refactor some existing questions:

• Currently we have "do you want to use Elasticsearch?" "do you want to use WebSockets?" etc...
• For all those tools, we could have instead a multi-check box question with "what additional tools do you want to configure?" which would include Elasticsearch, WebSockets, Kafka...
• Internally I would keep the current way of having booleans for using each technology (and not an Array of technologies, which would be more consistant), as it's much easier to use in switches in our code.

[deepu105]

👍 to refactor the questions. But we should keep the yo-rc format same as today.

Do you want to add Kafka to monoloth as well or just for microservices?

I can help you with refactoring the prompts part if you need.

I would merge the below into a single option with multiselect.

• enableSocialSignIn
• searchEngine
• clusteredHttpSession (does any one ever use this?)
• websocket

[xetys]

I am currently taking a look to Spring Cloud Bus, which is the default messaging solution for spring cloud microservices...so I would like to participate on this part, when I did some experiments with it.

One other hot topic is something called event sourcing, which in short is another way of service-to-service communication, as an allternate way to REST communication

[jdubois]

Great!

• Kafka is very good for microservices!! So I want it for monoliths, microservices and gateway. Shouldn't be much different.
• clustered HTTP sessions isn't widely used, and is generally a bad idea. But it's necessary in some situations, and very easy for us to keep. So I'd like to keep it. Now I have an horrible doubt: does this work with Undertow??? I totally forgot to test!
• Yes I'd like to keep the current .yo-rc.json if possible
• Yes your help in refactoring is welcome!

[codenullhackaton]

One alternative to clustered HTTP sessions is to use Spring-Session, it allows to cluster sessions in redis, jdbc, mongo, supports websockets, and I think supports undertow also

[devedipoint]

@xetys Regarding to event sourcing and whole JHipster's microservice approach I consider to align JHipster with Axon Framework - please look at it - as another approach to building microservice architecture with CQRS and DDD. Maybe it would be reasonable to align Spring Cloud Bus to Axon as a message bus and leverage whole Axon's CQRS?

[deepu105]

@jdubois OK then ill rewrite the promts first. you can add the kafka option to it later

[cbornet]

Aside from the docker config, what will we generate in the app ? Send the application logs and metrics for logstash ?

[jdubois]

For the moment, I would also add the dependencies, and configure the .yml files. Maybe we will need a specific Configuration class, but I'm not sure yet.

Nothing very complex, probably, so that would be small option, with little effect on the other options.

[jdubois]

Just commited a first version, no Gradle support yet.

[jdubois]

Just commited a new version with Gradle + microservices working.

• I will merge this branch into master and close this when it's done
• I will start doing documentation

[jdubois]

Closing as this is merged

[deepu105]

Should we add any default sample like we have the tracker for websocket?

[jdubois]

That's one of my goals, but not now:

• The front end is frozen until we have AngularJS 2
• I want to release a version early, and doing the whole sample is going to take time
• I don't have an idea yet for that sample!