search

graphite-project / docker-graphite-statsd

Official Docker image for Graphite
MIT License
473 stars 172 forks source link
docker graphite metrics statsd

readme

Docker Pulls Docker Size

This is official Graphite docker image repo.

This repo was based on @hopsoft's docker-graphite-statsd docker image and was used as base for "official" Graphite docker image with his permission. Also, it contains parts of famous @obfuscurity's synthesize Graphite installer. Thanks a lot, Nathan and Jason!

Any suggestions / patches etc. are welcome!

Tags / architectures history

Docker Image for Graphite & Statsd

Get Graphite & Statsd running instantly

Graphite & Statsd can be complex to setup. This image will have you running & collecting stats in just a few minutes.

Quick Start

docker run -d\
 --name graphite\
 --restart=always\
 -p 80:80\
 -p 2003-2004:2003-2004\
 -p 2023-2024:2023-2024\
 -p 8125:8125/udp\
 -p 8126:8126\
 graphiteapp/graphite-statsd

or

docker run -d\
 --name graphite\
 --restart=always\
 -p 80:80\
 -p 2003-2004:2003-2004\
 -p 2023-2024:2023-2024\
 -p 8125:8125/udp\
 -p 8126:8126\
 ghcr.io/deniszh/graphite-statsd

This starts a Docker container named: graphite

Please also note that you can freely remap container port to any host port in case of corresponding port is already occupied on host. It's also not mandatory to map all ports, map only required ports - please see table below.

That's it, you're done ... almost.

Includes the following components

Mapped Ports

Host Container Service
80 80 nginx
2003 2003 carbon receiver - plaintext
2004 2004 carbon receiver - pickle
2023 2023 carbon aggregator - plaintext
2024 2024 carbon aggregator - pickle
8080 8080 Graphite internal gunicorn port (without Nginx proxying).
8125 8125 statsd
8126 8126 statsd admin

By default, statsd listens on the UDP port 8125. If you want it to listen on the TCP port 8125 instead, you can set the environment variable STATSD_INTERFACE to tcp when running the container.

Please also note that you can freely remap container port to any host port in case of corresponding port is already occupied on host.

Mounted Volumes

Host Container Notes
DOCKER ASSIGNED /opt/graphite/conf graphite config
DOCKER ASSIGNED /opt/graphite/storage graphite stats storage
DOCKER ASSIGNED /opt/graphite/webapp/graphite/functions/custom graphite custom functions dir
DOCKER ASSIGNED /etc/nginx nginx config
DOCKER ASSIGNED /opt/statsd/config statsd config
DOCKER ASSIGNED /etc/logrotate.d logrotate config
DOCKER ASSIGNED /var/log log files
DOCKER ASSIGNED /var/lib/redis Redis TagDB data (optional)

Base Image

Version before (and including) 1.1.4-9 were built using Phusion's base image. Current version is based on Alpine Linux because of image size (please see PR#66 for details).

Start Using Graphite & Statsd

Send Some Stats

Let's fake some stats with a random counter to prove things are working.

while true; do echo -n "example:$((RANDOM % 100))|c" | nc -w 1 -u 127.0.0.1 8125; done

Visualize the Data

Open Graphite in a browser.

Secure the Django Admin

Update the default Django admin user account. The default is insecure.

First login at: http://localhost/account/login Then update the root user's profile at: http://localhost/admin/password_change/

Tunables

Additional environment variables can be set to adjust performance.

Graphite-web

TagDB

Graphite stores tag information in a separate tag database (TagDB). Please check tags documentation for details.

Collectd

Use COLLECTD=1 environment variable to enable local collectd instance

AMQP

Carbon-cache

If custom environment GRAPHITE_CARBONLINK_HOSTS variable is setup carbon-cache instances as daemons/services are managed based on the that, otherwise default instance (127.0.0.1:7002) is used.

Note: if default port 7002 is used among the hosts, need to setup CARBON_DISABLED=1 in the environment.

Carbon-relay

Use RELAY=1 environment variable to enable carbon relay instance. Use [relay] section of carbon.conf to configure it.

Note: in order to use carbon-relay daemon correctly, it must accept & distribute incoming traffic within DESTINATIONS endpoints which by default isn't (carbon-cache is). As one of solutions is to adjust graphitePort value to carbon-relay LINE_RECEIVER_PORT in statsd config.

Logrotate

By default logs are rotated daily, based on configuration in /etc/logrotate.d/graphite-statsd. You can variate rotation by adding size 10M or changing time up to hourly (instead of daily), because of logrotate is is evaluated hourly by /etc/periodic/hourly/logrotate. Please note, that according to Docker logging best practices "Ideally, applications log to stdout/stderr, and Docker sends those logs to the configured logging destination.". You can use - as log file name for such behaviour.

Runit

Each service started and controlled by runit will be gracefully shutdown when stopping the container : wait up to 7 seconds for the service to become down, then it will be killed. The runit environment variable $SVWAIT overrides this default timeout. Additionnally, a global timeout can be also specified with the docker-run option --stop-timeout. Each service started by default can be disabled by setting an environment variable named as : $<service name>_DISABLED. For instance : CARBON_AGGREGATOR_DISABLED=1, STATSD_DISABLED=1, etc. Please note, that any service in image can be disabled, so, some functionality can be broken in this case.

Startup custom scripts

At startup, entrypoint will run all scripts found in the directory /etc/run_once. It can be mounted with a docker-run option like this : --mount type=bind,source=/path/to/run_once,destination=/etc/run_once.

Change the Configuration

Read up on Graphite's post-install tasks. Focus on the storage-schemas.conf.

  1. Stop the container docker stop graphite.
  2. Find the configuration files on the host by inspecting the container docker inspect graphite.
  3. Update the desired config files.
  4. Restart the container docker start graphite.

Note: If you change settings in /opt/graphite/conf/storage-schemas.conf be sure to delete the old whisper files under /opt/graphite/storage/whisper/.

Important: Ensure your Statsd flush interval is at least as long as the highest-resolution retention. For example, if /opt/statsd/config/udp.js looks like this.

flushInterval: 10000

Ensure that storage-schemas.conf retentions are no finer grained than 10 seconds.

[all]
pattern = .*
retentions = 5s:12h # WRONG
retentions = 10s:12h # OK
retentions = 60s:12h # OK

Statsd Admin Management Interface

A management interface (default on port 8126) allows you to manage statsd & retrieve stats.

# show all current counters
echo counters | nc localhost 8126

More info & additional commands.

A Note on Volumes

You may find it useful to mount explicit volumes so configs & data can be managed from a known location on the host.

Simply specify the desired volumes when starting the container.

docker run -d\
 --name graphite\
 --restart=always\
 -v /path/to/graphite/configs:/opt/graphite/conf\
 -v /path/to/graphite/data:/opt/graphite/storage\
 -v /path/to/statsd_config:/opt/statsd/config\
 graphiteapp/graphite-statsd

Note: The container will initialize properly if you mount empty volumes at /opt/graphite/conf, /opt/graphite/storage, or /opt/statsd/config.

Memcached config

If you have a Memcached server running, and want to Graphite use it, you can do it using environment variables, like this:

docker run -d\
 --name graphite\
 --restart=always\
 -p 80:80\
 -p 2003-2004:2003-2004\
 -p 2023-2024:2023-2024\
 -p 8125:8125/udp\
 -p 8126:8126\
 -e "MEMCACHE_HOST=127.0.0.1:11211"\  # Memcached host. Separate by comma more than one servers.
 -e "GRAPHITE_DEFAULT_CACHE_DURATION=60"\              # in seconds
 graphiteapp/graphite-statsd

Also, you can specify more than one memcached server, using commas:

-e "MEMCACHE_HOST=127.0.0.1:11211,10.0.0.1:11211"

Running through docker-compose

The following command will start the graphite statsd container through docker-compose

docker-compose up

Running through Kubernetes

You can use this 3-rd party repo with Graphite Helm chart - https://github.com/kiwigrid/helm-charts/tree/master/charts/graphite

About root process

This image uses runit as init system, to run multiple processes in single container. It's not against Docker guidelines but bit against Docker philosophy. Also, runit require root privileges to run, so, it's not possible to stop using root privileges, without completely rewrite this image. This is possible, of course, but it's better to use separate images per component then, and having separate repository for this new project. Or you may use this image as base and create set of images to start Graphite components separately, without root access. Probably, result will be quite specific, but may be it's possible to make it generic enough to merge to this repo - in that case we would like to accept such PR.

Experimental Features

go-carbon

Use GOCARBON=1 environment variable to enable go-carbon instance instead of normal Carbon. Use GRAPHITE_CLUSTER_SERVERS="127.0.0.1:8000" if you want also use carbonserver feature.

brubeck

Use BRUBECK=1 environment variable to enable brubeck instance of normal Statsd. Please note that brubeck has different config format and not fully compatible with original statsd.

Additional Reading

Contributors

Build the image yourself.

  1. git clone https://github.com/graphite-project/docker-graphite-statsd.git
  2. docker build --build-arg python_binary=python3 -t graphiteapp/graphite-statsd .

    For using pypy instead of python3

  3. docker build --build-arg BASEIMAGE=jamiehewland/alpine-pypy:3.6-7.3-alpine3.11 --build-arg python_binary=/usr/local/bin/pypy3 -t graphiteapp/graphite-statsd .

Alternate versions can be specified via --build-arg:

Alternate repositories can also be specified with the build args graphite_repo, carbon_repo, whisper_repo & statsd_repo.

To build an image from latest graphite, whisper & carbon master, run:

docker build -t graphiteapp/graphite-statsd . --build-arg version=master --build-arg python_binary=python3

To build an image using a fork of graphite-web, run:

docker build -t forked/graphite-statsd . --build-arg version=master --build-arg graphite_repo=https://github.com/forked/graphite-web.git --build-arg python_binary=python3