Metrics

This repository contains scripts for running a full metrics stack for demonstration and end-to-end development.

Most of the other content of this repository has been split among other repositories hosted in the ArpNetworking and Inscope Metrics Github organizations.

Demo

THIS IS CURRENTLY NOT WORKING https://github.com/ArpNetworking/metrics/issues/31

The demo directory contains scripts for running a fully encapsulated end-to-end environment for demonstration purposes. All components are executed from binaries within a Vagrant controlled virtual machine. During provisioning the latest versions of Metrics Aggregator Daemon, Cluster Aggregator and Metrics Portal will be installed. If you wish to update these components in your demo environment, simply destroy the environment with vagrant destroy and then launch again.

To launch the stack execute:

demo> ./start.sh

To terminate the stack simply press Ctrl+c.

Options for start.sh are:

• -p -- run pinger
• -v -- verbose mode

The components are available at the following addresses:

• KairosDb: localhost:8082
• Grafana: localhost:8081
• Metrics Portal: localhost:8080

Build

The build directory contains scripts for running an end-to-end development environment for development of our components:

• Metrics Aggregator Daemon
• Cluster Aggregator Daemon
• Metrics Portal
• KairosDb Data Source

These development components are executed from source code directories expected as siblings to the metrics project directory (e.g. git clone into the same parent directory). The other three components:

• Cassandra
• KairosDb
• Grafana

Are executed from docker images on the local host. The Grafana and Cassandra use common public images. However, the KairosDb installation uses the KairosDb-Extensions generated image.

Ideally, we would build this from source as well.

To launch the stack execute:

build> ./start.sh -a

To terminate the stack simply press Ctrl+c.

Options for start.sh are:

• -s service -- specify a service to start; valid services are: mad, cagg, mportal, ckg
• -a -- start all services
• -t count -- start count instances of cagg from 1 to 3; default is 1
• -c -- clear logs
• -p -- run pinger
• -n -- don't build
• -v -- verbose mode

The components are available at the following addresses:

• KairosDb: localhost:8082
• Grafana: localhost:8081
• Metrics Portal: localhost:8080
• Metrics Aggregator Daemon: localhost:7090
• Cluster Aggregator:
  • Host 1: localhost:7068
  • Host 2: localhost:7070
  • Host 3: localhost:7072
• HAProxy:
  • Load Balanced Cluster Aggregator HTTP: localhost:7066
  • HAProxy Status: localhost:8066/stats

Component configuration is available in build/config/<component> and is automatically reloaded by Metrics Aggregator Daemon and Cluster Aggregator.

If you are iterating on one component of the stack, you should run start.sh twice to avoid restarting the entire stack with each change. For example, if you are iterating on Cluster Aggregator then start the rest of the stack first:

build> ./start.sh -s mad -s mportal -s ckg

Then start Cluster Aggregator:

build> ./start.sh -s cagg

Then after you make changes to Cluster Aggregator just shutdown the second start.sh process and restart it. By default start.sh will even ensure the project has been built! Of course, you can disable this behavior by adding the -n argument.

Logs for development each component are output to the logs directory in that component's project directory. The configuration for component logging is controlled from this project via the files in config/<component>/logback.xml.

Similarly, configuration of each component is found in config/<component>/.

Finally, all three development components are started with remote debugging enabled. The ports for each service are:

• Metrics Aggregator Daemon - 9001
• Metrics Portal - 9002
• Cluster Aggregator - 9101 through 9103 (depending on how many instances are launched)

Prerequisites

• Docker
• All compilation tools required by projects

FAQ

Q: What is the username and password to Grafana? A: The default username and password are both admin.

Q. Why does my dashboard json creation fail with "fieldNames":["Dashboard"],"classification":"RequiredError"? A. You need to wrap the dashboard json from the Grafana user interface with the following JSON:

{
  "overwrite": true,
  "dashboard": <JSON FROM UI HERE>
}

Q. Why does my dashboard json creation fail with "Dashboard not found","status":"not-found"? A. You need to remove the top-level "id":<INT> field from the dashboard json from the Grafana user interface.

Q. How do I get Cluster Aggregator to reaggregate data? A. Cluster Aggregator's persist vs reaggregate behavior is controlled by the endpoint that MAD sends data to. You can change this by swapping the active uri field in build/config/mad/pipelines/pipeline.conf for http_cluster_sink.

License

Published under Apache Software License 2.0, see LICENSE

© Inscope Metrics Inc., 2016