search

voxpupuli / puppet-collectd

Collectd module for Puppet
https://forge.puppet.com/puppet/collectd
Apache License 2.0
69 stars 272 forks source link
bsd-puppet-module centos-puppet-module debian-puppet-module freebsd-puppet-module gentoo-puppet-module hacktoberfest linux-puppet-module oraclelinux-puppet-module puppet redhat-puppet-module scientific-puppet-module ubuntu-puppet-module

readme

Collectd module for Puppet

Build Status Release Puppet Forge Puppet Forge - downloads Puppet Forge - endorsement Puppet Forge - scores puppetmodule.info docs Apache-2.0 License

Description

Puppet module for configuring collectd and plugins.

Usage

The simplest use case is to use all of the configurations in the default collectd.conf file shipped with collectd. This can be done by simply including the class:

include collectd

Collectd is most useful when configured with customized plugins. This is accomplished by removing the default collectd.conf file and replacing it with a file that includes all alternative configurations. Configure a node with the following class declaration:

class { 'collectd':
  purge           => true,
  recurse         => true,
  purge_config    => true,
  minimum_version => '5.4',
}

Set purge, recurse, and purge_config to true in order to override the default configurations shipped in collectd.conf and use custom configurations stored in conf.d. From here you can set up additional plugins as shown below.

Specifying the version or minimum_version of collectd as shown above reduces the need for two puppet runs to converge. See Puppet needs two runs to correctly write my conf, why? below.

Hiera example in YAML of passing install_options to the package resource for managing the collectd package. This parameter must be an array.

collectd::package_install_options:
  - '--nogpgcheck'

Simple Plugins

Example of how to load plugins with no additional configuration:

collectd::plugin { 'battery': }

Where battery is the name of the plugin.

Note: this should only be done in the case of a class for the plugin not existing in this module.

Repo management

The module will enable a repo by default.

On CentOS that will be EPEL:

On Ubuntu that'll be the CollectD PPA:

Public key keyserver

In case you need to change the server where to download the public key from for whatever reason (AKA: server is down) you can use the parameter $package_keyserver

CI Packages

Recently, Collectd CI packages are also avaliable from the CI repo

More information is avaliable here:

You can choose the CI repo with the $ci_package_repo parameter.

$ci_package_repo has to match '5.4', '5.5', '5.6', '5.7' or 'master' (RC for next release) as these are the current branches being built in the Collectd CI.

Configurable Plugins

Parameters will vary widely between plugins. See the collectd documentation for each plugin for configurable attributes.

Class: collectd::plugin::aggregation

collectd::plugin::aggregation::aggregator { 'cpu':
    plugin           => 'cpu',
    agg_type         => 'cpu',
    groupby          => ['Host', 'TypeInstance',],
    calculateaverage => true,
}

You can as well configure this plugin with a parameterized class :

class { 'collectd::plugin::aggregation':
  aggregators => {
    'cpu' => {
      plugin           => 'cpu',
      agg_type         => 'cpu',
      groupby          => ["Host", "TypeInstance",],
      calculateaverage => true,
    },
  },
}

Class: collectd::plugin::amqp

class { 'collectd::plugin::amqp':
  amqphost => '127.0.0.1',
  amqpvhost => 'myvirtualhost',
  graphiteprefix => 'collectdmetrics',
  amqppersistent => true,
}

Class: collectd::plugin::apache

class { 'collectd::plugin::apache':
  instances => {
    'apache80'     => {
      'url'      => 'http://localhost/mod_status?auto',
      'user'     => 'collectd',
      'password' => 'hoh2Coo6'
    },
    'lighttpd8080' => {
      'url' => 'http://localhost:8080/mod_status?auto'
    }
  },
}

Class: collectd::plugin::battery

class { 'collectd::plugin::battery':
  interval => 30,
  values_percentage => true,
  report_degraded => true,
  query_state_fs => true,
}

Class: collectd::plugin::bind

class { 'collectd::plugin::bind':
  url    => 'http://localhost:8053/',
}

Class: collectd::plugin::ceph

class { 'collectd::plugin::ceph':
  daemons        => [
    '[clustername]-osd.0',
    '[clustername]-osd.1',
    '[clustername]-osd.2',
    '[clustername]-mon.[hostname].asok'
  ],
  manage_package => true
}

Class: collectd::plugin::cgroups

See collectd plugin_cgroups documentation for more details.

class { 'collectd::plugin::cgroups':
  ignore_selected => true,
  cgroups         => ['array', 'of', 'paths']
}

Class: collectd::plugin::chain

class { 'collectd::plugin::chain':
    chainname     => "PostCache",
    defaulttarget => "write",
    rules         => [
      {
        'match'   => {
          'type'    => 'regex',
          'matches' => {
            'Plugin'         => "^cpu$",
            'PluginInstance' => "^[0-9]+$",
          },
        },
        'targets' => [
          {
            'type'       => "write",
            'attributes' => {
              "Plugin" => "aggregation",
            },
          },
          {
            'type' => "stop",
          },
        ],
      },
    ],
  }

Class: collectd::plugin::conntrack

class { 'collectd::plugin::conntrack':
}

Class: collectd::plugin::cpu

class { 'collectd::plugin::cpu':
  reportbystate => true,
  reportbycpu => true,
  valuespercentage => true,
}

Class: collectd::plugin::cpufreq

class { 'collectd::plugin::cpufreq':
}

Class: collectd::plugin::csv

class { 'collectd::plugin::csv':
  datadir    => '/etc/collectd/var/lib/collectd/csv',
  storerates => false,
}

Class: collectd::plugin::cuda

class { 'collectd::plugin::cuda':
}

Class: collectd::plugin::curl

collectd::plugin::curl::page {
  'stock_quotes':
    url      => 'http://finance.google.com/finance?q=NYSE%3AAMD',
    user     => 'foo',
    password => 'bar',
    matches  => [
      {
        'dstype'   => 'GaugeAverage',
        'instance' => 'AMD',
        'regex'    => ']*> *([0-9]*\\.[0-9]+) *',
        'type'     => 'stock_value',
      }],
}

You can as well configure this plugin with a parameterized class :

class { 'collectd::plugin::curl':
  pages => {
    'stock_GM' => {
      url      => 'http://finance.google.com/finance?q=NYSE%3AGM',
      user     => 'foo',
      password => 'bar',
      matches  => [
        {
          'dstype'   => 'GaugeAverage',
          'instance' => 'AMD',
          'regex'    => ']*> *([0-9]*\\.[0-9]+) *',
          'type'     => 'stock_value',
        },
      ],
    },
  },
}

Class: collectd::plugin::curl_json

collectd::plugin::curl_json {
  'rabbitmq_overview':
    url        => 'http://localhost:55672/api/overview',
    host       => 'rabbitmq.example.net',
    instance   => 'rabbitmq_overview',
    interval   => '300',
    user       => 'user',
    password   => 'password',
    digest     => 'false',
    verifypeer => 'false',
    verifyhost => 'false',
    cacert     => '/path/to/ca.crt',
    header     => 'Accept: application/json',
    post       => '{secret: \"mysecret\"}',
    timeout    => '1000',
    keys       => {
      'message_stats/publish' => {
        'type'     => 'gauge',
        'instance' => 'overview',
      },
    }
}

Class: collectd::plugin::dbi

collectd::plugin::dbi::database{'monitoring_node1':
  driver       => 'mysql',
  driveroption => {
    'host' => 'hostname',
    'username' => 'user',
    'password' => 'password',
    'dbname'   => 'monitoring'
  },
  query    => ['log_delay'],
}
collectd::plugin::dbi::query{'log_delay':
  statement => 'SELECT * FROM log_delay_repli;',
  results   => [{
    type           => 'gauge',
    instanceprefix => 'log_delay',
    instancesfrom  => 'inet_server_port',
    valuesfrom     => 'log_delay',
  }],
}

You can as well configure this plugin as a parameterized class :

class { 'collectd::plugin::dbi':
  package   => 'libdbd-mysql',
  databases => {
    'monitoring_node1' => {
      driver       => 'mysql',
      driveroption => {
        'host' => 'hostname',
        'username' => 'user',
        'password' => 'password',
        'dbname'   => 'monitoring'
      },
      query    => ['log_delay'],
    }
  },
}

Class: collectd::plugin::df

class { 'collectd::plugin::df':
  devices        => ['proc','sysfs'],
  mountpoints    => ['/u'],
  fstypes        => ['nfs','tmpfs','autofs','gpfs','proc','devpts'],
  ignoreselected => true,
}

Class: collectd::plugin::disk

class { 'collectd::plugin::disk':
  disks          => ['/^dm/'],
  ignoreselected => true,
  udevnameattr   => 'DM_NAME',
}

Class: collectd::plugin::dns

class { 'collectd::plugin::dns':
}

Parameters

See collectd documentation.

ensure

Optional. String that must be 'present' or 'absent'.

ignoresource

Optional. String of IP address to ignore.

interface

String of interface to use. May be interface identifier such as 'eth0' or 'any'.

interval

Numeric for interval in seconds. Changing this can be a disaster. Consult the documentation.

manage_package

Boolean to determine if system package for collectd's dns plugin should be managed. If set to true, you must specify the package name for any unsupported OS.

package_name

String for name of package. A value of 'USE_DEFAULTS' will set the value based on the platform. This is necessary when setting manage_package on an unsupported platform.

selectnumericquerytypes

Boolean for SelectNumericQueryTypes configuration option.

Class: collectd::plugin::dpdk_telemetry

class { 'collectd::plugin::dpdk_telemetry':
  client_socket_path => '/var/run/.client',
  dpdk_socket_path   => '/var/run/dpdk/rte/telemetry',
}

Class: collectd::plugin::dcpmm

class { 'collectd::plugin::dcpmm':
  interval             => 10.0,
  collect_health       => false,
  collect_perf_metrics => true,
  enable_dispatch_all  => false,
}

Class: collectd::plugin::entropy

class { 'collectd::plugin::entropy':
}

Class: collectd::plugin::exec

collectd::plugin::exec::cmd {
  'dummy':
    user => nobody,
    group => nogroup,
    exec => ["/bin/echo", "PUTVAL myhost/foo/gauge-flat N:1"],
}

You can also configure this plugin with a parameterized class:

class { 'collectd::plugin::exec':
  commands => {
    'dummy1' => {
      user  => nobody,
      group => nogroup,
      exec  => ["/bin/echo", "PUTVAL myhost/foo/gauge-flat1 N:1"],
    },
    'dummy2' => {
      user  => nobody,
      group => nogroup,
      exec  => ["/bin/echo", "PUTVAL myhost/foo/gauge-flat2 N:1"],
    },
  }
}

Class: collectd::plugin::ethstat

class { 'collectd::plugin::ethstat':
  interfaces => [ 'eth0', 'eth1'],
  maps       => [
    '"rx_csum_offload_errors" "if_rx_errors" "checksum_offload"', '"multicast" "if_multicast"'
  ],
  mappedonly => false,
}

Class: collectd::plugin::fhcount

class { 'collectd::plugin::fhcount':
  valuesabsolute   => true,
  valuespercentage => false,
}

Class: collectd::plugin::fscache

class { 'collectd::plugin::fscache':
}

Class: collectd::plugin::filecount

collectd::plugin::filecount::directory {'foodir':
  path          => '/path/to/dir',
  pattern       => '*.conf',
  mtime         => '-5m',
  recursive     => true,
  includehidden => false
}

You can also configure this plugin with a parameterized class:

class { 'collectd::plugin::filecount':
  directories => {
    'foodir' => {
      'path'          => '/path/to/dir',
      'pattern'       => '*.conf',
      'mtime'         => '-5m',
      'recursive'     => true,
      'includehidden' => false
      },
  },
}

For backwards compatibility:

class { 'collectd::plugin::filecount':
  directories => {
    'active'   => '/var/spool/postfix/active',
    'incoming' => '/var/spool/postfix/incoming'
  },
}

Class: collectd::plugin::filter

The filter plugin implements the powerful filter configuration of collectd. For further details have a look on the collectd manpage.

Parameters
Examples
Overwrite default chains
class { 'collectd::plugin::filter':
    ensure          => 'present',
    precachechain   => 'PreChain',
    postcachechain  => 'PostChain',
}
Full Example

This Example will rename the hostname of the mysql plugin.

include collectd::plugin::filter

# define default chains with default target
collectd::plugin::filter::chain { 'PreChain':
    target => 'return'
}
collectd::plugin::filter::chain { 'PostChain':
    target => 'write'
}

# create a third chain,
$chainname = 'MyAweseomeChain'
collectd::plugin::filter::chain { $chainname:
    ensure => present,
    target => 'return'
}

# add a new rule to chain
$rulename = 'MyAweseomeRule'
collectd::plugin::filter::rule { $rulename:
    chain => $chainname,
}

# add a new match rule, match metrics of the mysql plugin
collectd::plugin::filter::match { "Match mysql plugin":
    chain   => $chainname,
    rule    => $rulename,
    plugin  => 'regex',
    options => {
        'Plugin' => '^mysql',
    }
}

#rewrite hostname
collectd::plugin::filter::target{ "overwrite hostname":
    chain   => $chainname,
    rule    => $rulename,
    plugin  => 'set',
    options => {
        'Host' => 'hostname.domain',
    },
}

# hook the configured chain in the prechain
collectd::plugin::filter::target{ "1_prechain_jump_${chainname}":
    chain   => 'PreChain',
    plugin  => 'jump',
    options => {
        'Chain' => $chainname,
    },
}

Define: collectd::plugin::filter::chain

This define will create a new chain, which is required by targets, matches and rules.

Parameters
Example

see collectd::plugin::filter above

Define: collectd::plugin::filter::rule

This define will add a new rule to a specific chain

Parameters
Example

see collectd::plugin::filter above

Define: collectd::plugin::filter::target

This define will add a target to a chain or rule.

Parameters
Example

see collectd::plugin::filter above

Define: collectd::plugin::filter::match

This define will add a match rule.

Parameters
Example

see collectd::plugin::filter above

Class: collectd::plugin::genericjmx

include collectd::plugin::genericjmx

collectd::plugin::genericjmx::mbean {
  'garbage_collector':
    object_name     => 'java.lang:type=GarbageCollector,*',
    instance_prefix => 'gc-',
    instance_from   => ['name'],
    values          => [
      {
        mbean_type => 'invocations',
        table      => false,
        attribute  => 'CollectionCount',
      },
      {
        mbean_type      => 'total_time_in_ms',
        instance_prefix => 'collection_time',
        table           => false,
        attribute       => 'CollectionTime',
      },
    ];
}

collectd::plugin::genericjmx::connection {
  'java_app':
    host            => $fqdn,
    service_url     => 'service:jmx:rmi:///jndi/rmi://localhost:3637/jmxrmi',
    collect         => [ 'memory-heap', 'memory-nonheap','garbage_collector' ],
}

Class: collectd::plugin::hddtemp

class { 'collectd::plugin::hddtemp':
  host => '127.0.0.1',
  port => 7634,
}

Class: collectd::plugin::hugepages

class { 'collectd::plugin::hugepages':
  report_per_node_hp => true,
  report_root_hp     => true,
  values_pages       => true,
  values_bytes       => false,
  values_percentage  => false
}

Class: collectd::plugin::intel_pmu

class { 'collectd::plugin::intel_pmu':
  report_hardware_cache_events => true,
  report_kernel_pmu_events => true,
  report_software_events => true,
}

Class: collectd::plugin::mcelog

class { 'collectd::plugin::mcelog':
  mceloglogfile           => '/var/log/mcelog'
  memory                  => true
  mcelogclientsocket      => '/var/run/mcelog-client'
  persistentnotification  => true
}

Class: collectd::plugin::intel_rdt

class { 'collectd::plugin::intel_rdt':
  cores => ['0-2' '3,4,6' '8-10,15']
}

Class: collectd::plugin::interface

class { 'collectd::plugin::interface':
  interfaces     => ['lo'],
  ignoreselected => true
}

Class: collectd::plugin::irq

class { 'collectd::plugin::irq':
  irqs           => ['7', '23'],
  ignoreselected => true,
}

Class: collectd::plugin::ipc

class { 'collectd::plugin::ipc':
}

Class: collectd::plugin::ipmi

class { 'collectd::plugin::ipmi':
  ignore_selected           => true,
  sensors                   => ['temperature'],
  notify_sensor_add         => true,
  notify_sensor_remove      => true,
  notify_sensor_not_present => true,
}

Class: collectd::plugin::iptables

class { 'collectd::plugin::iptables':
  chains  => {
    'nat'    => 'In_SSH',
    'filter' => 'HTTP',
  },
  chains6 => {
    'filter' => 'HTTP6',
  },
}

Class: collectd::plugin::iscdhcp

class { 'collectd::plugin::iscdhcp': }

Class: collectd::plugin::java

jvmarg options must be declared if declaring loadplugin, as the JVM must be initialized prior to loading collectd java plugins.

class { 'collectd::plugin::java':
  jvmarg      => ['arg1', 'arg2'],
  loadplugin  => {"plugin.name" => ["option line 1", "option line 2"]}
}

Class: collectd::plugin::load

class { 'collectd::plugin::load':
}

Class: collectd::plugin::logfile

class { 'collectd::plugin::logfile':
  log_level => 'warning',
  log_file => '/var/log/collected.log'
}

Class: collectd::plugin::virt

The interface_format parameter was introduced in collectd 5.0 and will therefore be ignored (with a warning) when specified with older versions.

class { 'collectd::plugin::virt':
  connection       => 'qemu:///system',
  interface_format => 'address'
}

Class: collectd::plugin::lvm

class { 'collectd::plugin::lvm': }

Class: collectd::plugin::memcached

The plugin supports multiple instances specified via host+port and socket:

class { 'collectd::plugin::memcached':
  instances => {
    'sessions1' => {
      'host' => '192.168.122.1',
      'port' => '11211',
    },
    'storage1' => {
      'host' => '192.168.122.1',
      'port' => '11212',
    },
    'sessions2' => {
      'socket'  => '/var/run/memcached.sessions.sock',
    },
    'storage2' => {
      'socket'  => '/var/run/memcached.storage.sock',
    },
  }
}

Class: collectd::plugin::memory

class { 'collectd::plugin::memory':
}

Class: collectd::plugin::mysql

collectd::plugin::mysql::database { 'betadase':
  host        => 'localhost',
  username    => 'stahmna',
  password    => 'secret',
  port        => '3306',
  masterstats => true,
  wsrepstats  => true,
}

Class: collectd::plugin::mongodb

class { 'collectd::plugin::mongodb':
  db_user => 'admin',
  db_pass => 'adminpass',
}
Parameters
class { 'collectd::plugin::mongodb':
  db_host        => '127.0.0.1',
  db_user        => 'foo',
  db_pass        => 'bar',
  db_port        => '27017',
  configured_dbs => ['database', 'names'],
  collectd_dir   => '/collectd/module/path',
}

Class: collectd::plugin::netlink

class { 'collectd::plugin::netlink':
  interfaces        => ['eth0', 'eth1'],
  verboseinterfaces => ['ppp0'],
  qdiscs            => ['"eth0" "pfifo_fast-1:0"', '"ppp0"'],
  classes           => ['"ppp0" "htb-1:10"'],
  filters           => ['"ppp0" "u32-1:0"'],
  ignoreselected    => false,
}

Class: collectd::plugin::network

collectd::plugin::network::server{'hostname':
  port => 25826,
}

collectd::plugin::network::listener{'hostname':
  port => 25826,
}

You can as well configure this plugin with a parameterized class :

class { 'collectd::plugin::network':
  timetolive    => '70',
  maxpacketsize => '42',
  forward       => false,
  reportstats   => true,
  servers       => { 'hostname' => {
    'port'          => '25826',
    'interface'     => 'eth0',
    'securitylevel' => '',
    'username'      => 'foo',
    'password'      => 'bar',},
  },
  listeners     => { 'hostname' => {
    'port'          => '25826',
    'interface'     => 'eth0',
    'securitylevel' => '',
    'authfile'      => '/etc/collectd/passwd',},
  },
}

Class: collectd::plugin::nfs

class { 'collectd::plugin::nfs':
}

Class: collectd::plugin::nginx

class { 'collectd::plugin::nginx':
  url      => 'https://localhost:8433',
  user     => 'stats',
  password => 'uleePi4A',
}

Class: collectd::plugin::ntpd

class { 'collectd::plugin::ntpd':
  host           => 'localhost',
  port           => 123,
  reverselookups => false,
  includeunitid  => false,
}

Class: collectd::plugin::numa

class { 'collectd::plugin::numa':
}

Class: collectd::plugin::nut

class { 'collectd::plugin::nut':
    upss => [ 'ups@localhost:port' ]
}

Class: collectd::plugin::openldap

class { 'collectd::plugin::openldap':
  instances => {
    'foo' => {
      'url' => 'ldap://localhost/'
    },
    'bar' => {
      'url' => 'ldaps://localhost/'
    }
  },
}

Class: collectd::plugin::openvpn

Watch multiple statusfiles:

class { 'collectd::plugin::openvpn':
  statusfile             => [
    '/etc/openvpn/openvpn-status-tcp.log',
    '/etc/openvpn/openvpn-status-udp.log'
  ],
  collectindividualusers => false,
  collectusercount       => true,
}

Watch the single default statusfile:

class { 'collectd::plugin::openvpn':
  collectindividualusers => false,
  collectusercount       => true,
}

Class: collectd::plugin::pcie_errors

class { 'collectd::plugin::pcie_errors':
  source                   => undef,
  access_dir               => undef,
  report_masked            => false,
  persistent_notifications => false,
}

Class: collectd::plugin::perl

This class has no parameters and will load the actual perl plugin. It will be automatically included if any perl::plugin is defined.

Example
include collectd::plugin::perl

Define: collectd::plugin::perl::plugin

This define will load a new perl plugin.

Parameters
Examples
Using a preinstalled plugin
collectd::plugin::perl::plugin { 'foo':
    module          => 'Collectd::Plugins::Foo',
    enable_debugger => "",
    include_dir     => '/usr/lib/collectd/perl5/lib',
}
Using a plugin from a file from source
collectd::plugin::perl::plugin { 'baz':
    module      => 'Collectd::Plugins::Baz',
    provider    => 'file',
    source      => 'puppet:///modules/myorg/baz_collectd.pm',
    destination => '/path/to/my/perl5/modules'
}
Using a plugin from cpan (requires the puppet cpan module)
collectd::plugin::perl::plugin {
  'openafs_vos':
    module        => 'Collectd::Plugins::OpenAFS::VOS',
    provider      => 'cpan',
    source        => 'Collectd::Plugins::OpenAFS',
    config        => {'VosBin' => '/usr/afsws/etc/vos'},
}
Using a plugin from package source
collectd::plugin::perl::plugin {
  'bar':
    module        => 'Collectd::Plugins::Bar',
    provider      => 'package',
    source        => 'perl-Collectd-Plugins-Bar',
    config        => {'foo' => 'bar'},
}

Class: collectd::plugin::ping

class { 'collectd::plugin::ping':
    hosts => ['example.com'],
}

Class: collectd::plugin::postgresql

collectd::plugin::postgresql::database{'monitoring_node1':
  name     => 'monitoring',
  port     => '5433',
  instance => 'node1',
  host     => 'localhost',
  user     => 'collectd',
  password => 'collectd',
  query    => 'log_delay',
}
collectd::plugin::postgresql::query{'log_delay':
  statement => 'SELECT * FROM log_delay_repli;',
  params    => ['database'],
  results   => [{
    type           => 'gauge',
    instanceprefix => 'log_delay',
    instancesfrom  => 'inet_server_port',
    valuesfrom     => 'log_delay',
  }],
}
collectd::plugin::postgresql::writer{'sqlstore':
  statement  => 'SELECT collectd_insert($1, $2, $3, $4, $5, $6, $7, $8, $9);',
  storerates => 'true',
}

You can as well configure this plugin as a parameterized class :

class { 'collectd::plugin::postgresql':
  databases => {
    'postgres' => {
      'host'     => '/var/run/postgresql/',
      'user'     => 'postgres',
      'password' => 'postgres',
      'sslmode'  => 'disable',
      'query'    => [ 'query_plans', 'queries', 'table_states', 'disk_io' ],
    },
    'devdb' => {
      'host'     => 'host.example.com',
      'port'     => '5432',
      'user'     => 'postgres',
      'password' => 'secret',
      'sslmode'  => 'prefer',
    }
  }
}

Class: collectd::plugin::powerdns

You can either specify powerdns servers / recursors at once:

class { 'collectd::plugin::powerdns':
  recursors => {
    'recursor1' => {
      'socket'  => '/var/run/my-socket',
      'collect' => ['cache-hits', 'cache-misses'],
    },
    'recursor2' => {}
  },
  servers => {
    'server1' => {
      'socket'  => '/var/run/my-socket',
      'collect' => ['latency', 'recursing-answers', 'recursing-questions'],
    }
  },
}

Or define single server / recursor:

collectd::plugin::powerdns::recursor { 'my-recursor' :
  socket  => '/var/run/my-socket',
  collect => ['cache-hits', 'cache-misses'],
}
collectd::plugin::powerdns::server { 'my-server' :
  socket  => '/var/run/my-socket',
  collect => ['latency', 'recursing-answers', 'recursing-questions'],
}

Class: collectd::plugin::processes

You can either specify processes / process matches at once:

class { 'collectd::plugin::processes':
  processes => ['process1', 'process2'],
  process_matches => [
    { name => 'process-all', regex => 'process.*' }
  ],
}

Or define single processes / process matches:

collectd::plugin::processes::process { 'collectd' : }
collectd::plugin::processes::processmatch { 'elasticsearch' :
  regex => '.*java.*org.elasticsearch.bootstrap.Elasticsearch'
}

Class: collectd::plugin::protocols

class { 'collectd::plugin::protocols':
  values => ['/^Tcp:*/', '/^Udp:*/', 'Icmp:InErrors' ],
  ignoreselected => false,
}

Class: collectd::plugin::python

The plugin uses a fact python_dir to find the python load path for modules. python or python3 must be installed as a pre-requisite for the this fact to give a non-default value.

NOTE: Since v3.4.0 the syntax of this plugin has changed. Make sure to update your existing configuration. Now you can specify multiple Python modules at once:

class { 'collectd::plugin::python':
  modulepaths => ['/usr/share/collectd/python'],
  modules     => {
    'elasticsearch' => {
      'script_source' => 'puppet:///modules/myorg/elasticsearch_collectd_python.py',
      'config'        => [{'Cluster' => 'elasticsearch'},],
    },
    'another-module' => {
      'config'        => [{'Verbose' => 'true'},],
    }
  }
  logtraces   => true,
  interactive => false,
}

When script_source provided, a file called {module}.py will be created in $modulepath/$module.py.

Or define single module:

collectd::plugin::python::module {'zk-collectd':
  script_source => 'puppet:///modules/myorg/zk-collectd.py',
  config        => [
    {'Hosts'   => "localhost:2181",
     'Verbose' => true,
     'Values'  => ["abc","def"],
     'Name'    => 'My Name',
     'Limit'   => 4.5,
    }
  ]
}

The resulting configuration would be

Import "zk-collectd"
<Module "zk-collectd">
  Hosts "localhost:2181"
  Verbose true
  Values "abc" "def"
  Limit 4.5
</Module>

Each plugin might use different modulepath, however make sure that all paths are included in collectd::plugin::python variable modulepaths. If no modulepath is specified, OS default will be used.

collectd::plugin::python::module {'my-module':
  modulepath    => '/var/share/collectd',
  script_source => 'puppet:///modules/myorg/my-module.py',
  config        => [
    {'Key'   => "value",
     'Value' => 3.4,
    }
  ]
}

Class: collectd::plugin::redis

class { 'collectd::plugin::redis':
  nodes => {
    'node1' => {
      'host'     => 'localhost',
    },
    'node2' => {
      'host'     => 'node2.example.com',
      'port'     => '6380',
      'timeout'  => 3000,
    },
    'node3' => {
      'host'    => 'node3.example.com',
      'queries' => {
          'dbsize' => {
              'type'  => 'count',
              'query' => 'DBSIZE',
          },
      },
    },
  }
}

Class: collectd::plugin::rabbitmq

Please note the rabbitmq plugin provides a types.db.custom. You will need to add this to collectd::config::typesdb via hiera or in a manifest. Failure to set the types.db.custom content will result in no metrics from the rabbitmq plugin.

The rabbitmq plugin has not been ported to python3 and will fail on CentOS 8 #75

set typesdb to include the collectd-rabbitmq types.db.custom

collectd::config::typesdb:
  - /usr/share/collectd/types.db
  - /usr/share/collect-rabbitmq/types.db.custom
class { 'collectd::plugin::rabbitmq':
  config           => {
    'Username' => 'admin',
    'Password' => $admin_pass,
    'Scheme'   => 'https',
    'Port'     => '15671',
    'Host'     => $facts['fqdn'],
    'Realm'    => 'RabbitMQ Management',
  },
}

Class: collectd::plugin::rrdcached

class { 'collectd::plugin::rrdcached':
  daemonaddress => 'unix:/var/run/rrdcached.sock',
  datadir       => '/var/lib/rrdcached/db/collectd',
}

Class: collectd::plugin::rrdtool

class { 'collectd::plugin::rrdtool':
  datadir           => '/var/lib/collectd/rrd',
  createfilesasync  => false,
  rrarows           => 1200,
  rratimespan       => [3600, 86400, 604800, 2678400, 31622400],
  xff               => 0.1,
  cacheflush        => 900,
  cachetimeout      => 120,
  writespersecond   => 50
}

Class: collectd::plugin::sensors

class {'collectd::plugin::sensors':
  sensors        => ['sensors-coretemp-isa-0000/temperature-temp2', 'sensors-coretemp-isa-0000/temperature-temp3'],
  ignoreselected => false,
}

Class: collectd::plugin::smart

class { 'collectd::plugin::smart':
  disks          => ['/^dm/'],
  ignoreselected => true,
}

Class: collectd::plugin::snmp

class {'collectd::plugin::snmp':
  data  =>  {
    amavis_incoming_messages => {
      'type'         => 'counter',
      'table'        => false,
      'instance'     => 'amavis.inMsgs',
      'values'       => ['AMAVIS-MIB::inMsgs.0'],
      'ignore'       => [ '00:00', '*IgnoreString' ],
      'invert_match' => false,
    }
  },
  hosts => {
    debianvm => {
      'address'   => '127.0.0.1',
      'version'   => 2,
      'community' => 'public',
      'collect'   => ['amavis_incoming_messages'],
      'interval'  => 10
    }
  },
}
class { 'collectd::plugin::snmp':
  data  => {
    hc_octets => {
      'type'     => 'if_octets',
      'table'    => true,
      'instance' => 'IF-MIB::ifName',
      'values'   => ['IF-MIB::ifHCInOctets', 'IF-MIB::ifHCOutOctets'],
    },
  },
  hosts => {
    router => {
      'address'            => '192.0.2.1',
      'version'            => 3,
      'security_level'     => 'authPriv',
      'username'           => 'collectd',
      'auth_protocol'      => 'SHA',
      'auth_passphrase'    => 'mekmitasdigoat',
      'privacy_protocol'   => 'AES',
      'privacy_passphrase' => 'mekmitasdigoat',
      'collect'            => ['hc_octets'],
      'interval'           => 10,
    },
  },
}

Class: collectd::plugin::snmp_agent

class {'collectd::plugin::snmp_agent':
  table => {
    ifTable => {
      'indexoid' => 'IF-MIB::ifIndex',
      'sizeoid' => 'IF-MIB::ifNumber',
      data => [{
        ifDescr => {
          'plugin' => 'interface',
          'oids' => 'IF-MIB::ifDescr'
        },
        'ifDescr2' => {
          'plugin' => 'interface2',
          'oids' => 'IF-MIB::ifDescr2'
        }
      }]
    }
  },
  data => {
    memAvailReal => {
      'plugin' => 'memory',
      'type' => 'memory',
      'oids' => '1.3.6.1.4.1.2021.4.6.0',
      'typeinstance' => 'free',
      'indexkey' => {
      'source' => 'PluginInstance'
      }
    }
  }
}

Class: collectd::plugin::statsd

class { 'collectd::plugin::statsd':
  host            => '0.0.0.0',
  port            => 8125,
  deletecounters  => false,
  deletetimers    => false,
  deletegauges    => false,
  deletesets      => false,
  timerpercentile => ['50','90'],
}

Class: collectd::plugin::swap

class { 'collectd::plugin::swap':
  reportbydevice => false,
  reportbytes    => true
}

Class: collectd::plugin::syslog

class { 'collectd::plugin::syslog':
  log_level => 'warning'
}

Class: collectd::plugin::target_v5upgrade

class { 'collectd::plugin::target_v5upgrade':
}

Class: collectd::plugin::tcpconns

class { 'collectd::plugin::tcpconns':
  localports  => ['25', '12026'],
  remoteports => ['25'],
  listening   => false,
}

Class: collectd::plugin::tail

collectd::plugin::tail::file { 'exim-log':
  filename => '/var/log/exim4/mainlog',
  instance => 'exim',
  matches  => [
    {
      regex    => 'S=([1-9][0-9]*)',
      dstype   => 'CounterAdd',
      type     => 'ipt_bytes',
      instance => 'total',
    },
    {
      regex    => '\\<R=local_user\\>',
      dstype   => 'CounterInc',
      type     => 'counter',
      instance => 'local_user',
    }
  ]
}

Class: collectd::plugin::tail_csv

class { 'collectd::plugin::tail_csv':
  metrics => {
    'snort-dropped' => {
      'type'        => 'gauge',
      'values_from' => 1,
      'instance'    => "dropped"
    },
  },
  files  => {
    '/var/log/snort/snort.stats' => {
      'collect'   => ['snort-dropped'],
      'plugin'    => 'snortstats',
      'instance'  => 'eth0',
      'interval'  => 600,
      'time_from' => 5,
    }
  }
}

Class: collectd::plugin::thermal

class { 'collectd::plugin::thermal':
  devices        => ['foo0'],
  ignoreselected => false,
}

Class: collectd::plugin::threshold

class { 'collectd::plugin::threshold':
  hosts   => [
    {
      name    => 'example.com',
      plugins => [
        {
          name  => 'load',
          types => [
            {
              name        => 'load',
              data_source => 'shortterm',
              warning_max => $facts.dig('processors', 'count') * 1.2,
              failure_max => $facts.dig('processors', 'count') * 1.9,
            },
            {
              name        => 'load',
              data_source => 'midterm',
              warning_max => $facts.dig('processors', 'count') * 1.1,
              failure_max => $facts.dig('processors', 'count') * 1.7,
            },
            {
              name        => 'load',
              data_source => 'longterm',
              warning_max => $facts.dig('processors', 'count'),
              failure_max => $facts.dig('processors', 'count') * 1.5,
            },
          ],
        },
      ],
    },
  ],
  plugins => [
    # See plugin definition above
  ],
  types   => [
    # See types definition above
  ],
}

Class: collectd::plugin::unixsock

class {'collectd::plugin::unixsock':
  socketfile   => '/var/run/collectd-sock',
  socketgroup  => 'nagios',
  socketperms  => '0770',
  deletesocket => false,
}

Class: collectd::plugin::uptime

class {'collectd::plugin::uptime':
}

Class: collectd::plugin::users

class {'collectd::plugin::users':
}

Class: collectd::plugin::uuid

class {'collectd::plugin::uuid':
  uuid_file => '/etc/uuid',
}

Class: collectd::plugin::varnish

class { 'collectd::plugin::varnish':
  instances => {
    'instanceName' => {
      'CollectCache' => 'true',
      'CollectBackend' => 'true',
      'CollectConnections' => 'true',
      'CollectSHM' => 'true',
      'CollectESI' => 'false',
      'CollectFetch' => 'true',
      'CollectHCB' => 'false',
      'CollectTotals' => 'true',
      'CollectWorkers' => 'true',
    }
  },
}

Class: collectd::plugin::vmem

class { 'collectd::plugin::vmem':
  verbose => true,
}

Class: collectd::plugin::write_graphite

The write_graphite plugin writes data to Graphite, an open-source metrics storage and graphing project.

collectd::plugin::write_graphite::carbon {'my_graphite':
  graphitehost   => 'graphite.example.org',
  graphiteport   => 2003,
  graphiteprefix => '',
  protocol       => 'tcp'
}

You can define multiple Graphite backends where will be metrics send. Each backend should have unique title:

collectd::plugin::write_graphite::carbon {'secondary_graphite':
  graphitehost        => 'graphite.example.org',
  graphiteport        => 2004,
  graphiteprefix      => '',
  protocol            => 'udp',
  escapecharacter     => '_',
  alwaysappendds      => true,
  storerates          => true,
  separateinstances   => false,
  logsenderrors       => true
  preserveseparator   => false,
  dropduplicatefields => false,
}

The preserveseparator and dropduplicatefields fields are supported as of collectd version >= 5.7.

Class: collectd::plugin::write_http

The write_http plugin supports two ways of configuration, the old plugin format using urls:

class { 'collectd::plugin::write_http':
  urls => {
    'collect1.example.org' => { 'format' => 'JSON' },
    'collect2.example.org' => {},
  }
}

And the new plugin format using nodes:

class { 'collectd::plugin::write_http':
  nodes => {
    'collect1' => { 'url' => 'collect1.example.org', 'format' => 'JSON' },
    'collect2' => { 'url' => 'collect2.example.org'},
  }
}

Class: collectd::plugin::write_kafka

class { 'collectd::plugin::write_kafka':
  kafka_host => 'localhost',
  kafka_port => 9092,
  topics     => {
    'mytopic'      => { 'format' => 'JSON' },
  },
  properties => {
    'myproperty'   => { 'myvalue' },
  },
  meta       => {
    'mymeta'       => { 'myvalue' },
  }
}

Class: collectd::plugin::write_log

class { 'collectd::plugin::write_log':
  format => 'JSON',
}

Class: collectd::plugin::write_prometheus

class { 'collectd::plugin::write_prometheus':
  host => 'localhost',
  port => '9103',
}

Note: Requires collectd 5.7 or later.

Class: collectd::plugin::write_network

Deprecated
class { 'collectd::plugin::write_network':
  servers => {
    'collect1.example.org' => { 'serverport' => '25826' },
    'collect2.example.org' => { 'serverport' => '25826' }
  }
}

Class: collectd::plugin::write_riemann

class { 'collectd::plugin::write_riemann':
  nodes => [
    {
      'name' => 'riemann.example.org',
      'host' => 'riemann.example.org',
      'port' => 5555,
      'protocol' => 'TCP'
    }
  ],
  tags         => ['foo'],
  attributes   => {'bar' => 'baz'},
}

Class: collectd::plugin::write_sensu

class { 'collectd::plugin::write_sensu':
  sensu_host => 'sensu.example.org',
  sensu_port => 3030,
}

Class: collectd::plugin::write_tsdb

class { 'collectd::plugin::write_tsdb':
  host             => 'tsdb.example.org',
  port             => 4242,
  host_tags        => ['environment=production', 'colocation=AWS'],
  store_rates      => false,
  always_append_ds => false,
}

Class: collectd::plugin::zfs_arc

class { 'collectd::plugin::zfs_arc':
}

Class: collectd::plugin::zookeeper

class { 'collectd::plugin::zookeeper':
  zookeeper_host  => 'localhost',
  zookeeper_port  => '2181',
}
types.db

Collectd needs to know how to handle each collected datapoint. For this it uses a database file called types.db

Those files can be created using the collectd::typesdb and collectd::type define resources.

$db = '/etc/collectd/types.db'
collectd::typesdb { $db: }

collectd::type { "response_size-${db}":
  target  => $db,
  ds_type => 'ABSOLUTE',
  min     => 0,
  max     => 10000000,
  ds_name => 'value',
}

class { 'collectd':
  typesdb      => [
    '/usr/share/collectd/types.db',
    $typesdb,
  ],
}

Other software may need to read the Collectd types database files. To allow non-root users to read from a collectd::typesdb file like so:

$db = '/etc/collectd/types.db'
collectd::typesdb { $db:
  mode => '0644',
}

Puppet Tasks

Assuming that the collectdctl command is available on remote nodes puppet tasks exist to run collectdctl and collect results from nodes. The tasks rely on python3 being available also.

Puppet Task collectd::listval

$ bolt task show collectd::listval
collectd::listval - Lists all available collectd metrics

USAGE:
bolt task run --nodes <node-name> collectd::listval
collectd::listval - Lists all available collectd metrics

USAGE:
bolt task run --nodes <node-name> collectd::listval

Puppet Task collectd::getval

$ bolt task show collectd::getval

collectd::getval - Get a particular metric for a host

USAGE:
bolt task run --nodes <node-name> collectd::getval metric=<value>

PARAMETERS:
- metric: String[1]
    Name of metric, e.g. load/load-relative

Example Task collectd::getval

$ bolt -u root task run collectd::getval metric=load/load-relative -n aiadm32.example.org

returns the values of the load metric.

  {
    "metric": "load/load-relative",
    "values": {
      "shortterm": "1.750000e-01",
      "longterm": "8.000000e-02",
      "midterm": "8.500000e-02"
    }
  }

Limitations

See metadata.json for supported platforms

Known issues

Puppet needs two runs to correctly write my conf, why?

Some plugins will need two runs of Puppet to fully generate the configuration for collectd. See this issue. This can be avoided by specifying a minimum version ($minimum_version) for the collectd class. e.g. Setting this to 1.2.3 will make this module assume on the first run (when the fact responsible to provide the collectd version is not yet available) that your systems are running collectd 1.2.3 and generate the configuration accordingly.

Development

Running tests

This project contains tests for rspec-puppet.

Quickstart:

gem install bundler
bundle install
bundle exec rake lint
bundle exec rake validate
bundle exec rake rubocop
bundle exec rake spec SPEC_OPTS='--format documentation'

Version scoping

Some plugins or some options in plugins are only available for recent versions of collectd.

This module shall not use unsupported configuration directives. Look at templates/loadplugin.conf.erb for a hands-on example.

Please make use of the search by branch/tags on the collectd github to see when a function has been first released.

Reading the collectd.conf.pod file is good, validating the presence of the code in the .c files is even better.

Authors

Puppet-collectd is maintained by VoxPupuli. Before moving to VoxPupuli it was written and maintained by TheCAT in the pdxcat github org.