search

CiscoCloud / kubernetes-ansible

Install and configure Google Kubernetes on OpenStack
Apache License 2.0
82 stars 37 forks source link

Alternative logging/monitoring #31

Open altvnk opened 9 years ago

altvnk commented 9 years ago

Make possible to forward logs and metrics in customer specific location

chrholme commented 8 years ago

I've been doing some thinking regarding this requirement and the requirements outlined at https://github.com/CiscoCloud/cis-paas/blob/master/docs/design/mantl/logging.md I think the simplest approach would be to pass messages through a queue such that they can be easily fanned out to customer specific locations.
I would think Kafka would be a good choice as it aligns with the plans in MI. I'm just a little concerned with some of the reading I've been doing regarding Zookeeper stability in K8S. Specifically: https://github.com/kubernetes/kubernetes/issues/5017 There's also a lack of TLS for Kafka which might raise Infosec concerns.
I noticed we're looking at bring Sensu in for Monitoring; which would bring RabbitMQ for its transport layer. Rabbit has native support for TLS and no requirement for ZK. I would be interested in opinions on its suitability as a log message queue.

rramchan55 commented 8 years ago

Seems like there has been done SSL support implemented recently in Kafka

https://issues.apache.org/jira/plugins/servlet/mobile#issue/KAFKA-1690

If there is a work around to the issues with running kakfa on kubernetes, we could go ahead with Kafka instead of RabbitMQ.

I've been doing some thinking regarding this requirement and the requirements outlined at https://github.com/CiscoCloud/cis-paas/blob/master/docs/design/mantl/logging.md I think the simplest approach would be to pass messages through a queue such that they can be easily fanned out to customer specific locations.

I would think Kafka would be a good choice as it aligns with the plans in MI. I'm just a little concerned with some of the reading I've been doing regarding Zookeeper stability in K8S. Specifically: kubernetes/kubernetes#5017 https://github.com/kubernetes/kubernetes/issues/5017 There's also a lack of TLS for Kafka which might raise Infosec concerns.

I noticed we're looking at bring Sensu in for Monitoring; which would bring RabbitMQ for its transport layer. Rabbit has native support for TLS and no requirement for ZK. I would be interested in opinions on its suitability as a log message queue.

— Reply to this email directly or view it on GitHub https://github.com/CiscoCloud/kubernetes-ansible/issues/31#issuecomment-138674080 .