Kubernetes support: cassandra, elk, consul

pascalgrimaud commented 8 years ago

As suggested on Twitter (@deepu105, @PierreBesson, @saturnism), Kubernetes support need to be improved:

[X] npm test
[X] elasticsearch -> I already coded this _elasticsearch.yml file but currently it isn't included to the kubernetes sub generator
[ ] cassandra -> on local, it's easy by mounting volume to our cql files to create all tables, but I have no idea how it should work with Kubernetes. Maybe manually? Or using Pet Set as suggested by @saturnism
[ ] ELK with JHipster console
[X] kafka -> very new, from the latest release 3.8.0
[x] consul -> the PR from @PierreBesson need to be merged first

Feel free to code one part of this, if someone is motivated :star:

pascalgrimaud commented 8 years ago

I'm working first on npm test for kubernetes

PierreBesson commented 8 years ago

I think we should use Helm and officially supported charts, this will make our job easier: https://github.com/kubernetes/charts They already have config for mysql, mariadb, mongodb, elasticsearch, consul.

pascalgrimaud commented 8 years ago

I'm working on elasticsearch @PierreBesson : https://github.com/kubernetes/charts seems interesting but I don't see how it can be used with our subgenerator.

raphaelbrugier commented 8 years ago

I'm interested in helping with Cassandra and Kafka. Can you please tell me where to start or describe more the problems?

PierreBesson commented 8 years ago

Hi @raphaelbrugier, similar to a what's already in place in the docker-compose subgen we want to have a multi-node Cassandra that works out of the box with a JHipster app and that can scale at will. One resource that might help you is the 1000 cassandra node demo using JHipster :

https://github.com/k8s-for-greeks/gpmr

Good luck !

PierreBesson commented 8 years ago

raphaelbrugier commented 8 years ago

Awesome! Thanks, I'll read that and play a bit with k8s as a start.

pascalgrimaud commented 8 years ago

@raphaelbrugier thanks !!

you can begin with kafka, it should be easier
then, about cassandra, it should be more complex because you can't mount volume as we do locally. Maybe start 1st the cassandra cluster, launch cql manually, then launch all other apps? I don't have idea about this, at the moment
you can use minikube locally (that's what I do because I don't have k8s) to test your configuration
the sub generator kubernetes uses same functions from sub generator docker-compose

Don't hesitate to ping me if you need help

raphaelbrugier commented 8 years ago

Thanks @pascalgrimaud, I'll start with Kafka then. I have just installed minikube and started playing with k8s, so I need to get familiar with it first.

Does the hacktoberfest tag mean you'll owe me a beer for my contribution? ;)

pascalgrimaud commented 8 years ago

@raphaelbrugier sure, with pleasure !!:beers:

raphaelbrugier commented 8 years ago

So, after playing with the JHipster k8s generator and researching on how to integrate Kafka I have a couple of questions.

1) What are the features that the configuration generated by JHipster should cover? Production ready? high availability? fault tolerant? Or is it just for local testings with minikube and to give examples of a kubernetes configuration?

I am asking, because when I tested with a single JHipster app and a mysql database, the webapp was correctly fault tolerant (I killed the container and k8s restarted a new one as expected) but the mysql container was not. All the tables and data were gone. I may be wrong somewhere in my tests because the k8s mysql service declares a volume data which is supposed to survive a container failure (if restarted in the same pod).

2) Kafka is not currently "supported" by kubernetes By supported, I mean there is no official example or charts unlike for Cassandra.

An example for Zookeeper using petset has just landed in the official repository. I have found this work in progress but I have not tested it yet.

Here are my options, also depending on 1)

do not try to cover HA and write something very simple (one ZK node, one Kafka node) but that works out of the box locally
dig more and write our own Kakfa config using petset
wait for an example on the official repository

What is your point of view on this?

pascalgrimaud commented 7 years ago

@raphaelbrugier :

Note: everything here is my point of view

Docker-compose subgen generates docker-compose yaml files. k8s subgen should do the same, with the appropriate syntax.

IMO, both (k8s and dockercompose subgenerator) should provide some basic configuration, then the user can custom it, accordingly to what he needs. It's like the java code => JHipster helps people to start quickly, then the user has to custom!
about volume with mysql (or something else), I didn't have a look on how volumes work with k8s, so I have no idea
about kafka: something very basic and simple, like you said, 1 ZK node, 1 Kafka node => it should be to transform kafka.yml (for docker-compose) into kafka.yml (for k8s)

Last word about volume:

The current configuration for kafka doesn't need volume so that's why it should not be very difficult.
About ELK, Cassandra, Consul, if you look on the compose files, all of them use volume. So it's a little bit more complex.

PierreBesson commented 7 years ago

@rafaelbrugier You make really good points. Configuring "prod" deployments of things like Cassandra or Zookeeper is very hard. This is what the DataMC team is doing at Ippon for their Cassandra and Kafka managed services offering and it's a full time job. Anyway it would be lying to imply that we provide a complete prod setup here. A simple deployment or petset should be enough to help users get started.

raphaelbrugier commented 7 years ago

Hi,

Thank you both for your answers.

Ok, let's keep it simple and just add a warning in the documentation page to explain it's up to the users to tweak for their needs (more fault tolerant == more complicated).

I'll just port the Kafka docker-compose file to k8s.

@pascalgrimaud About volumes:

I don't know about ELK and Consul but for Cassandra the volume section in the docker-compose file generated by JHipster is commented. Were you referring to something else?

@PierreBesson Julien demoed us DataMC last week and that was pretty impressive! Good job guys!

pascalgrimaud commented 7 years ago

@raphaelbrugier : I was refering to this https://github.com/jhipster/generator-jhipster/blob/master/test/templates/compose/05-cassandra/src/main/docker/cassandra-migration.yml#L12-L13

With Kubernetes, we have probably to find another way to play CQL scripts

raphaelbrugier commented 7 years ago

Oh thanks Pascal, I forgot about this one :). Yes, we'll have to rethink how to apply the CQL scripts when deploying in k8s. I have always preferred the idea to have the deployment pipeline responsible for applying the scripts, so it really depends on what we want to achieve with the k8s default setup.

I have not make any progress on Kafka because I'm running into a problem with the existing k8s configuration for a monolithic/mysql JHipster app. The fist time I used the generated configuration and deployed it in minikube, it worked fine. Now, when I try to use it again, the JHipster app complains it can not reach the mysql address:

Caused by: java.net.UnknownHostException: jhipsterapp-mysql.default.svc.cluster.local: unknown error

If I change the webapp container configuration (SPRING_DATASOURCE_URL) to point directly to the IP of the MySQL container, it works.

I have tried to recreate everything from scratch, deleting the .minikube folder and recreating the JHipster application, but the same problem occurred.

I'm not sure of the reason, probably a DNS problem? Any idea?

pascalgrimaud commented 7 years ago

Can you look at :

Maybe it's related to this issue

EDIT: oh I read too fast. It's not related to kafka. I'll have a look tomorrow: monolithic+mysql, so ?

pascalgrimaud commented 7 years ago

@raphaelbrugier : I retry to deploy a simple monolith application. Everything works fine on my minikube

raphaelbrugier commented 7 years ago

Thanks @pascalgrimaud, Yes, my problem is with a very simple JHipster monolith application from scratch with just a MySQL database. It was working fine the first time I used the k8s generator, but I doest not work anymore.

I'll give it another try tomorrow with a fresh install of minikube and the latest JHipster master.

pascalgrimaud commented 7 years ago

I begin to work on consul

pascalgrimaud commented 7 years ago

Edit: I only begin to write tests for consul, but I don't have time to finish it. I will do it later -> as I will help for testing angular2

pascalgrimaud commented 7 years ago

@PierreBesson : we need to test https://github.com/kubernetes-incubator/kompose :-)

PierreBesson commented 7 years ago

Yes I have heard that it has improved a lot recently and now supports docker-compose v2.

jdubois commented 7 years ago

@pascalgrimaud @PierreBesson is there still work to do here? Nothing on this ticket since October, and Cassandra support isn't very important (only very few people use it, less than 2%)

pascalgrimaud commented 7 years ago

I think we can close it, as there are few feedback on Kubernetes. And it can be done later! Nothing here is urgent.

jhipster / generator-jhipster

Kubernetes support: cassandra, elk, consul #4205