search

cloudstax / firecamp

Serverless Platform for the stateful services
https://www.cloudstax.io
Apache License 2.0
209 stars 20 forks source link

Add kafka-manager #25

Closed jazzl0ver closed 6 years ago

jazzl0ver commented 6 years ago

Please, implement kafka-manager (https://github.com/yahoo/kafka-manager) into Kafka service creation. A ready-to-use docker image - https://hub.docker.com/r/sheepkiller/kafka-manager/ - can be used. Probably the best way is to add a command line keys like -enable-kafka-manager and -kafka-manager-port=9000 to run it along with the kafka service.

JuniusLuo commented 6 years ago

kafka-manager is a good project. Will further evaluate how to add it.

JuniusLuo commented 6 years ago

The initial Kafka Manager service is added. We decided to add as a separate service, as one Kafka Manager could manage multiple Kafka services. Please take a look and see if it works for you. Thanks!

jazzl0ver commented 6 years ago

It does work, awesome! Thanks a bunch! One thing I noticed is that the log group for kafkamanager looks like firecamp-firecamp-qa-kafkamanager-qa-b52..., while my other loggroups looks like firecamp-qa-kafka-qa-ae9a... Could you please fix loggroup name generation? The command line for kafkamanager creation was:

./firecamp-service-cli -cluster=firecamp-qa -region=us-east-1 -op=create-service -service-type=kafkamanager -service-name=kafkamanager-qa -km-heap-size=512 -km-zk-service=zoo-qa
jazzl0ver commented 6 years ago

Please, add Kafka Manager readme to the tutorials wiki page

JuniusLuo commented 6 years ago

yes, the log group format is firecamp-clustername-servicename-serviceuuid. Your cluster name is "firecamp-qa", so the log group name will start with firecamp-firecamp-qa.

JuniusLuo commented 6 years ago

Thanks for the remind! Added to wiki.

jazzl0ver commented 6 years ago

yes, the log group format is firecamp-clustername-servicename-serviceuuid. Your cluster name is "firecamp-qa", so the log group name will start with firecamp-firecamp-qa.

it used to be different before, right? the point is that i name my clusters like 'firecamp-uat' or 'firecamp-qa' to differentiate them from other clusters like 'uat' and 'qa' where the apps are running. could you please revert that back and name log groups after the cluster name w/o any prefixes? this is really more logical than the current implementation (imho).

JuniusLuo commented 6 years ago

yes, the firecamp prefix is added recently. sounds a good suggstion. we could revert it.

JuniusLuo commented 6 years ago

The fix was committed.