Hephy Logger

Hephy - A Fork of Deis Workflow

Deis (pronounced DAY-iss) Workflow is an open source Platform as a Service (PaaS) that adds a developer-friendly layer to any Kubernetes cluster, making it easy to deploy and manage applications on your own servers.

For more information about the Deis Workflow, please visit the main project page at https://github.com/teamhephy/workflow.

We welcome your input! If you have feedback, please submit an issue. If you'd like to participate in development, please read the "Development" section below and submit a pull request.

Description

A system logger for use in the Deis Workflow open source PaaS.

This Docker image is based on quay.io/deis/base image. You can see what version we are currently using in the Dockerfile

The new v2 logger implementation has seen a simplification from the last rewrite. While it still uses much of that code it no longer depends on etcd. Instead, we will use kubernetes service discovery to determine where logger is running.

We have also decided to not use logspout as the mechanism to get logs from each container to the logger component. Now we will use fluentd which is a widely supported logging framework with hundreds of plugins. This will allow the end user to configure multiple destinations such as Elastic Search and other Syslog compatible endpoints like papertrail.

Configuration

The following environment variables can be used to configure logger:

Development

The only assumption this project makes about your environment is that you have a working docker host to build the image against.

Building binary and image

To build the binary and image run the following make command:

DEV_REGISTRY=quay.io IMAGE_PREFIX=myaccount make build
IMAGE_PREFIX=myaccount make build
DEV_REGISTRY=myhost:5000 make build

Pushing the image

The makefile assumes that you are pushing the image to a remote repository like quay or dockerhub. So you will need to supply the REGISTRY environment variable.

DEV_REGISTRY=quay.io IMAGE_PREFIX=myaccount make push
IMAGE_PREFIX=myaccount make push
DEV_REGISTRY=myhost:5000 make push

Kubernetes interactions

• make install - Install the recently built docker image into the kubernetes cluster
• make upgrade - Upgrade a currently installed image
• make uninstall - Uninstall logger from a kubernetes cluster

Architecture Diagram

                        ┌────────┐
                        │ Router │                  ┌────────┐
                        └────────┘                  │ Logger │
                            │                       └────────┘
                        Log file                        │
                            │                           │
                            ▼                           ▼
┌────────┐             ┌─────────┐    logs/metrics   ┌─────┐
│App Logs│──Log File──▶│ fluentd │───────topics─────▶│ NSQ │
└────────┘             └─────────┘                   └─────┘
                                                        │
                                                        │
┌─────────────┐                                         │
│ HOST        │                                         ▼
│  Telegraf   │───┐                                ┌────────┐
└─────────────┘   │                                │Telegraf│
                  │                                └────────┘
┌─────────────┐   │                                    │
│ HOST        │   │    ┌───────────┐                   │
│  Telegraf   │───┼───▶│ InfluxDB  │◀────Wire ─────────┘
└─────────────┘   │    └───────────┘   Protocol
                  │          ▲
┌─────────────┐   │          │
│ HOST        │   │          ▼
│  Telegraf   │───┘    ┌──────────┐
└─────────────┘        │ Grafana  │
                       └──────────┘