search

jhipster / generator-jhipster

JHipster is a development platform to quickly generate, develop, & deploy modern web applications & microservice architectures.
https://www.jhipster.tech
Apache License 2.0
21.53k stars 4.02k forks source link

Add cache replication configuration #3137

Closed jdubois closed 8 years ago

jdubois commented 8 years ago

This is mostly for JHipster 3.0 with microservices, when you want to scale your nodes.

Currently ehcache only works in "standalone" mode, so if you scale your services you will get cache synchronization issues.

Please note we have the same issue with Hazelcast, but I'm focusing on ehcache here as it's the default cache, and it's what the vast majority of people use. Once this is done, we could open a similar ticket with Hazelcast.

What needs to be done, only for application type "microservice" (it might be interesting for other application types, but first we focus on the main issue):

deepu105 commented 8 years ago

@jdubois why do you think hazelcast has the issue? it is clustered by deafult and should work out of the box. I have done both ehcache and hazelcast and and Im happy with how hazelcast works in clusters compared to ehcache which is not realtime sync, I have tried the ehcache synchronization and it wasnt that great. For micro services I would suggest making hazelcast as the cache option as it would auto discover and cluster out of the box

deepu105 commented 8 years ago

Also doing with hazelcast would be much easier as normally it will autodiscover if the same application is found in the network and we can even restrict that by IP with a single line in yaml

jdubois commented 8 years ago

Oh, if that's a lot simpler with Hazelcast we can start with it, and make it the default for microservices... Let's do a first test with Hazelcast before.

deepu105 commented 8 years ago

Actually hazelcast will make more sense for microservices, Our default hazelcast config worked for me with an app running on 4 nodes on the same network.

The only issue I had was when I ran dev and staging app on same network as they joined clusters due to auto discovery so I just added a IP whitelisting config to hazelcast in cache config.

may be I can add the same config, its few lines, to our microservices as well if required

jdubois commented 8 years ago

Oh if you already know the config, and even have it running in prod, can you do this? I'll rename the ticket accordingly!

deepu105 commented 8 years ago

Yup ill give it a try On 11 Mar 2016 19:02, "Julien Dubois" notifications@github.com wrote:

Oh if you already know the config, and even have it running in prod, can you do this? I'll rename the ticket accordingly!

— Reply to this email directly or view it on GitHub https://github.com/jhipster/generator-jhipster/issues/3137#issuecomment-195321756 .

jdubois commented 8 years ago

Thanks @noctarius who pointed me to exactly what I want with Hazelcast - here is the code.

So the idea is that:

This wouldn't use multi-cast, and would use our "normal" Eureka registry, so this would work much better with our current architecture.

@noctarius can you help us with that code? We would make Hazelcast our default cache for microservices, of course.

noctarius commented 8 years ago

Eureka discovery doesn't work yet, trying to build it based on Eureka 2 (RC2), anyhow the examples seem to be outdated (from master) - still fighting :)

If anybody has experience with Eureka 1 I'm happy to do a discovery plugin based on the older version as well. Generally speaking, building a discovery plugin based on Hazelcast 3.6, it basically is 3 classes and you're done.

Happy to help out on integrating Hazelcast seamlessly here. Maybe it makes sense to use JCache as the general abstraction between JHipster and Hazelcast / EHcache - maybe there's another abstraction already in place - not very knowledgable about the internal design, sorry :D

jdubois commented 8 years ago
jdubois commented 8 years ago

Guys, I've got something running which is much, much simpler :-) @deepu105 have you started working on it? Otherwise just wait a little bit until my code is ready, I'll commit it directly

deepu105 commented 8 years ago

Nope I haven't started yet. Plz do even I was wondering why it wouldn't work simply out of the box using the default discovery mechanism On 13 Mar 2016 21:35, "Julien Dubois" notifications@github.com wrote:

Guys, I've got something running which is much, much simpler :-) @deepu105 https://github.com/deepu105 have you started working on it? Otherwise just wait a little bit until my code is ready, I'll commit it directly

— Reply to this email directly or view it on GitHub https://github.com/jhipster/generator-jhipster/issues/3137#issuecomment-195958392 .

jdubois commented 8 years ago

I got the cluster running, and it looks all easy & simple, but my entities are not in sync between my cluster nodes. As the cluster looks fine I'm wondering if we don't have another, older issue. Do you have this working fine in multicast ?

deepu105 commented 8 years ago

Yes I didn't have any issues reported in my prod apps so far On 13 Mar 2016 22:28, "Julien Dubois" notifications@github.com wrote:

I got the cluster running, and it looks all easy & simple, but my entities are not in sync between my cluster nodes. As the cluster looks fine I'm wondering if we don't have another, older issue. Do you have this working fine in multicast ?

— Reply to this email directly or view it on GitHub https://github.com/jhipster/generator-jhipster/issues/3137#issuecomment-195965033 .

jdubois commented 8 years ago

OK I got it working locally (on "localhost"). I'm pretty confident my configuration is also good with a real cluster, and I'm committing it as tomorrow I'll have a full cluster to test this :-) So we should know quickly if it works in all situations!

deepu105 commented 8 years ago

Ill test it as well, im gonna do a microservice demo day after tomorrow as well On 14 Mar 2016 01:44, "Julien Dubois" notifications@github.com wrote:

Closed #3137 https://github.com/jhipster/generator-jhipster/issues/3137 via 86d3c03 https://github.com/jhipster/generator-jhipster/commit/86d3c03cf689d6191afd176311f9f92f5ced179e .

— Reply to this email directly or view it on GitHub https://github.com/jhipster/generator-jhipster/issues/3137#event-587899060 .

deepu105 commented 8 years ago

Thats very similar to what i did to restrict IPs except I knew the ip so had them in yml beforehand On 14 Mar 2016 10:39, "Deepu K Sasidharan" d4udts@gmail.com wrote:

Ill test it as well, im gonna do a microservice demo day after tomorrow as well On 14 Mar 2016 01:44, "Julien Dubois" notifications@github.com wrote:

Closed #3137 https://github.com/jhipster/generator-jhipster/issues/3137 via 86d3c03 https://github.com/jhipster/generator-jhipster/commit/86d3c03cf689d6191afd176311f9f92f5ced179e .

— Reply to this email directly or view it on GitHub https://github.com/jhipster/generator-jhipster/issues/3137#event-587899060 .

deepu105 commented 8 years ago

I have made hazelcast default when choosing gateway or microservice

noctarius commented 8 years ago

Does anybody wants to give this one a shot: https://github.com/hazelcast-incubator/eurekast-one

Simple test is working but I guess you guys know way more about eureka than I do :)