CacheServerApplication unable to connect to Gemfire cluster running apart from localhost

[nayakmk]

Hi @jxblum,

Can you please provide minimum configuration needed to connect a Gemfire CacheServerApplication as a REPLICATE_PROXY to a Gemfire/Geode cluster running apart from localhost ?

I have a cluster running which I am able to connect from ClientCacheApplication but the same configuration is not working for CacheServerApplication.

Regards, Malaya

[jxblum]

Hi @nayakmk - The main bit to connect a GemFire/Geode CacheServer application configured and bootstrapped with Spring is to set the (GemFire/Geode) locators property.

The first thing to keep in mind is, you should not confuse the Pool locators attribute with the locators GemFire/Geode property (search for "locators" in the "Settings" column). These are not the same thing.

The former (i.e. Pool locators attribute) is used by cache clients (i.e. ClientCache instances that are configured and bootstrapped with SDG's @ClientCacheApplication annotation) to connect to a GemFire/Geode cluster of servers. This is modeled in SDG Annotation-based configuration using the @ClientCacheApplication.locator attribute as well as the @ClientCacheApplication.Locator type. You should review the ref doc for more details. I recommend configuration properties whenever/wherever possible.

The later (i.e. GemFire/Geode locators property) is used by nodes (e.g. servers) that a user wants connected to a cluster as a peer member (i.e. Cache instance or Locator instance, in other words, not a client). The GemFire/Geode locators property is not used by clients at all.

As you know, there are 2 ways to configure and bootstrap a peer member of a GemFire/Geode cluster using SDG. First is by using the @PeerCacheApplication annotation and the second is with the @CacheServerApplication annotation. Both create a Cache instance, peer node in the GemFire/Geode cluster, but @CacheServerApplication additionally creates Sockets on which cache clients (i.e. ClientCache instances) can connect.

NOTE: Clients cannot connect to a normal, data hosting peer Cache node that is not also serving clients. To serve clients, or ClientCache instances, that requires 1 or more embedded CacheServer instances.

NOTE: Gfsh starts all servers as CacheServers so that client can connect by default, unless the --disable-default-server attribute is specified to the start server command.

As such, both SDG's @PeerCacheApplication (here) and @CacheServerApplication (here) annotations have a locators attribute.

You can either set this in code, or as I would recommend, using the property spring.data.gemfire.locators property.

This property takes the format of: hostOne[portOne],hostTwo[portTwo],...,hostN[portN] where host# is the IP address or the DNS network name of the machine hosting the Locator(s) and the port# is the port on which the Locator running at that particular host is listening.

If this property (or annotation attribute) is properly set, then the Spring configured/bootstrapped @CacheServerApplication should join an (existing) cluster, even if the other Locators/Servers were started with Gfsh.

The Spring Boot for Apache Geode (SBDG) reference documentation is a bit more organized and detailed where configuring and bootstrapping servers are concerned. For example, on "Building Embedded (Peer & Server) Cache Applications".

This chapter even demonstrates configuring and bootstrapping multiple CacheServers with Spring (SBDG/SDG) as well as Gfsh, all connected together as peers in a single cluster.

Hope this helps!

-John

[jxblum]

In summary (and, in short), you could do something along the following lines:

@CacheServerApplication(name="ServerOne", port=12345, locators="hostOne{10335],hostTwo[10335]")
class MySpringDataGeodeCacheServerApplication {

    @Bean("MyReplicateRegion")
    public ReplicatedRegionFactoryBean<Object, Object> replicateRegion(GemFireCache gemfireCache) {

        ReplicatedRegionFactoryBean<Object, Object> replicateRegion = new ReplicatedRegionFactoryBean<>();

        replicateRegion.setCache(gemfireCache);
        replicateRegion.setreplicateRegion.setShortcut(RegionShortcut.REPLICATE_PROXY);

        return replicateRegion;
    }
}

The name of the REPLICATE Region must match all servers hosting the same REPLICATE Region, whether a proxy or not.

The "name" of the Region is the same as the bean name (e.g. "MyReplicateRegion") unless you configure it otherwise.

Keep in mind that you must also vary the servers name (absolutely, when the server is in the same cluster) as well as the CacheServer port number if multiple instances of the MySpringDataGeodeCacheServerApplication class is bootstrapped on the same machine.

Node names in GemFire/Geode must be unique, which is a GemFire/Geode requirement.

It is recommended to set both of these annotation attributes using properties instead.

spring.data.gemfire.name=ServerOne
spring.data.gemfire.cache.server.port=41414

Alternatively, you can set the CacheServer port to the ephemeral port (0), which is useful during (integration) testing.

[nayakmk]

Thanks for the detailed explanation as always. Indeed I was confused with those two locator properties. I will try out this and update in case of issues.

On the servers point, if I have multiple spring server applications (pods) connected to a cluster and a cachewriter or listener is configured then will it notify all the servers or any one ? My Q is mainly coming from the write behind scenario.

[jxblum]

On the servers point, if I have multiple spring server applications (pods) connected to a cluster and a cachewriter or listener is configured then will it notify all the servers or any one ? My Q is mainly coming from the write behind scenario.

That is a good question, and I don't remember what happens off the top of my head for a REPLICATE_PROXY.

[nayakmk]

One more thing I noticed is that, the version of cluster running geode and the spring data geode version, needs to be compatible if connecting as Server application, along with the cluster Java version and Server application Java version. Is this right understanding ?

[jxblum]

This doc describes the event handling for REPLICATE Regions (scroll down) in general.

However, this is what I was specifically looking for...

https://geode.apache.org/docs/guide/114/developing/region_options/data_hosts_and_accessors.html

I would say that PROXY type Regions (whether REPLICATE or PARTITION) should receive events and if a CacheCallback was configured/registered (e.g. CacheListener or CacheWriter) then it should be called, even for proxies (??).

Of course, this a GemFire/Geode behavior that has nothing to do with Spring. Spring can help you wire the CacheCallback to the [*_PROXY] Region accordingly though.

[jxblum]

The version of cluster running geode and the spring data geode version, needs to be compatible if connecting as Server application, along with the cluster Java version and Server application Java version. Is this right understanding ?

Well, this has more to do with the Geode bits and specifically, protocols between peers (TCP, UDP, etc), or even clients and servers for that matter, than anything Spring does/does not do.

Clients have the only major restriction that I know of. That is, older clients can talk to same or newer servers, but newer clients cannot talk to older servers.

AFAIK, although don't quote me on it, older and newer GemFire/Geode servers can generally play together, hence GemFire/Geode's rolling upgrade feature.

[jxblum]

Also, just a quick and friendly reminder, this type of question is more aptly asked on StackOverflow. We can continue the conversation to completion here, this time around.

Always happy to help. ;-)

[nayakmk]

Sorry for the deviation from the main problem statement. I have got all my answers and will utilise SOF next time. For others to find the discussion will create a Q and A there with this reference.

Thanks you for your response and in case of any further questions I will connect via SOF.

Regards, Malaya

[jxblum]

No problem! Again, happy to help. Wishing you all the best.

[nayakmk]

I can confirm now, that with the locator property change it is able to connect to cluster. Thanks again.