GenMetrics

Runtime metrics for GenServer and GenStage applications.

Important! The GenMetrics library is not suitable for use within long-running production environments. For further details, see the benchmarks performance guide.

This library supports the collection and publication of GenServer and GenStage runtime metrics. Metrics data are generated by an introspection agent. No instrumentation is required within the GenServer or GenStage library or within your application source code.

By default, metrics are published by a dedicated GenMetrics reporting process. Any application can subscribe to this process in order to handle metrics data at runtime. Metrics data can also be pushed directly to a statsd agent which makes it possible to analyze, and visualize the metrics within existing tools and services like Graphana and Datadog.

Quick Look: GenServer Metrics

Given an application with the following GenServers: Session.Server, Logging.Server, activate metrics collection for the server cluster as follows:

alias GenMetrics.GenServer.Cluster
cluster = %Cluster{name: "demo",
                   servers: [Session.Server, Logging.Server],
                   opts: [window_interval: 5000]}
GenMetrics.monitor_cluster(cluster)

Metrics are published by a dedicated GenMetrics reporting process. Any application can subscribe to this process in order to receive metrics data. Sample summary metrics data for a GenServer process looks as follows:

# Server Name: Demo.Server, PID<0.176.0>

%GenMetrics.GenServer.Summary{name: Demo.Server,
                              pid: #PID<0.176.0>,
                              calls: 8000,
                              casts: 34500,
                              infos: 3333,
                              time_on_calls: 28,
                              time_on_casts: 161,
                              time_on_infos: 15}

# Summary timings measured in milliseconds (ms).

Detailed statistical metrics data per process are also available. See the documentation for details.

Quick Look: GenStage Metrics

Given a GenStage application with the following stages: Data.Producer, Data.Scrubber, Data.Analyzer and a Data.Consumer, activate metrics collection for the entire pipeline as follows:

alias GenMetrics.GenStage.Pipeline
pipeline = %Pipeline{name: "demo",
                     producer: [Data.Producer],
                     producer_consumer: [Data.Scrubber, Data.Analyzer],
                     consumer: [Data.Consumer]}
GenMetrics.monitor_pipeline(pipeline)

# Stage Name: Data.Producer, PID<0.195.0>

%GenMetrics.GenStage.Summary{stage: Data.Producer,
                             pid: #PID<0.195.0>,
                             callbacks: 9536,
                             time_on_callbacks: 407,
                             demand: 4768000,
                             events: 4768000}

# Summary timings measured in milliseconds (ms).

Detailed statistical metrics data per process are also available. See the documentation for details.

Quick Look: GenMetrics Sampling

Given an application with the following GenServers: Session.Server, Logging.Server, activate metrics-sampling for the server cluster as follows:

alias GenMetrics.GenServer.Cluster
cluster = %Cluster{name: "demo",
                   servers: [Session.Server, Logging.Server],
                   opts: [sample_rate: 0.3]}
GenMetrics.monitor_cluster(cluster)

Given a GenStage application with the following stages: Data.Producer, Data.Scrubber, Data.Analyzer and a Data.Consumer, activate metrics-sampling for the entire pipeline as follows:

alias GenMetrics.GenStage.Pipeline
pipeline = %Pipeline{name: "demo",
                     producer: [Data.Producer],
                     producer_consumer: [Data.Scrubber, Data.Analyzer],
                     consumer: [Data.Consumer],
                     opts: [sample_rate: 0.1]}
GenMetrics.monitor_pipeline(pipeline)

Quick Look: Metrics Reporting

Redirect your GenServer cluster metrics data to the Datadog service as follows:

alias GenMetrics.GenServer.Cluster
cluster = %Cluster{name: "demo",
                   servers: [Session.Server, Logging.Server],
                   opts: [statistics: :datadog]}
GenMetrics.monitor_cluster(cluster)

Redirect your GenStage pipeline metrics data to a statsd agent as follows:

alias GenMetrics.GenStage.Pipeline
pipeline = %Pipeline{name: "demo",
                     producer: [Data.Producer],
                     producer_consumer: [Data.Scrubber, Data.Analyzer],
                     consumer: [Data.Consumer],
                     opts: [statistics: :statsd]}
GenMetrics.monitor_pipeline(pipeline)

Documentation

Find detailed documentation for the GenMetrics library on HexDocs.

Installation

GenStage requires Elixir v1.4. Just add :gen_metrics to your list of dependencies in mix.exs:

def deps do
  [{:gen_metrics, "~> 0.3.0"}]
end

Benchmarks

For those of you curious about the performance impact gen_metrics has on the servers and pipelines it is monitoring, we've put together a number of benchmarks along with a detailed performance analysis which you can find here.

Examples

Examples using GenMetrics to collect and report runtime metrics for GenServer applications can be found in the examples directory:

genserver_events

Examples using GenMetrics to collect and report runtime metrics for GenStage applications can also be found in the examples directory:

All of these GenStage example applications are clones of the example applications provided in the GenStage project repository.

License

See the LICENSE file for license rights and limitations (Apache License 2.0).

onetapbeyond / gen_metrics

readme