cloudstax / firecamp

Serverless Platform for the stateful services
https://www.cloudstax.io
Apache License 2.0
209 stars 20 forks source link

Cassandra/Kafka usage question #7

Closed jazzl0ver closed 6 years ago

jazzl0ver commented 6 years ago

Hello. Not sure it's a good place to ask questions, but you might move this Q/A to the wiki section if you find it relevant. What is the best way to get connected to the services like Cassandra/Kafka from other applications (Java code, for example)? I need to give my developers IP addresses of the service endpoints, but since the EC2 instances might be terminated and started new ones at any time, that IP addresses will be changed and an application will lose access to the service.

JuniusLuo commented 6 years ago

Every service member will get a unique dns name. For example, the cluster is testcluster, Cassandra service name is mycas with 3 replicas. FireCamp will assign the dns name, mycas-0.testcluster-firecamp.com, mycas-1.testcluster-firecamp.com, and mycas-2.testcluster-firecamp.com, to the 3 Cassandra containers. The applications could simply access the Cassandra cluster via these 3 dns names.

When the EC2 instance goes down, let's say the EC2 of mycas-1.testcluster-firecamp.com, AutoScaleGroup will start a new EC2. ECS will schedule the Cassandra container to the new EC2. FireCamp will attach the original EBS volume that belongs to mycas-1.testcluster-firecamp.com, and update the address of mycas-1.testcluster-firecamp.com to the new EC2's privateIP in Route53.

If the applications use JVM, you will need to set JVM TTL. Please check Cassandra/Kafka readme, catalog/cassandra/README.md. "By default, JVM caches a successful DNS lookup forever. If you use Cassandra Java CQL driver, please set JVM TTL to a reasonable value such as 60 seconds. So when Cassandra container moves to another node, Java CQL driver could lookup the new address."

cloudstax commented 6 years ago

added the explanation to readme.