search

BrightcoveOS / Diamond

1.18k stars 384 forks source link

Question: collectors config ? #51

Closed tomplus closed 12 years ago

tomplus commented 12 years ago

I suppose that in conf/collectors were example config files, but now this directory is empty and is listed in .gitignore ... I think it's mistake.

kormoc commented 12 years ago

It's there to allow per collector config overrides to be placed and scanned for. Realistically all we need to do is get setup.py to actually just create the directory as a blank directory rather then have it on the filesystem.

tomplus commented 12 years ago

Ok, but I think it'd be useful to see example config to my favorite collector without reading manual (or source).

Maybe we can create structure like Apache (mods-enable/available)

/etc/graphite/collectors-available/ ... (xyz.conf.example) /etc/graphite/colectors-enabled/

rafalio commented 12 years ago

I agree that this kind of collectors structure would be very nice.

I started using Diamond couple days ago and I'm very confused as to how to enable/disable collectors, so if anyone could shed some light on this I would be grateful. For example, I see this

# All collectors are disabled by default
enabled = False

in my conf file, yet graphite is still receiving lots of data regarding cpu, load averages, and other things.

I would be very grateful if someone could write a part of the wiki on enabling/disabling individual collectors.

kormoc commented 12 years ago

That's in there as the default, but a few basic collectors override the master value to be enabled. I'll look into simplifying this and getting a wiki page written up, but the simple way is to add a specific collector.conf file or a specific section to the master file per collector to enable/disable them.

Samuel29 commented 12 years ago

IMHO, the best approach is to provide us with each collectors' configuration file, containing default values.

1st workaround : disable everything then see what you need to enable. this small shell command might help begginers like me to get rid of all the noise : (use with caution, all previous configs will be lost)

for i in $(grep -r 'class .Collector' /usr/share/diamond/collectors/ | grep -v 'class Test' | cut -d'(' -f1 | cut -d' ' -f2); do echo "enabled = False" > /etc/diamond/collectors/$i.conf ; done

kormoc commented 12 years ago

@Samuel29 , Right now, if you run make config, it'll ask you some basic questions and generate configuration files. I'm hoping to expand this functionality soon.

Samuel29 commented 12 years ago

@kormoc thanks for the precision, which deserves a small paragraph in the wiki pages :) but, AFAIK this method will not include those files in a .deb package ? (ie. make builddeb)

kormoc commented 12 years ago

@Samuel29 It actually will. If you generate configuration files and then run make builddeb or make buildrpm it will package the configuration into the package for deployment.

Samuel29 commented 12 years ago

@kormoc Awesome ! In fact I have built the .deb once then included it into an amazon image, so I didn't need to build it anymore.

Anyway, I'll remember this thanks !

kormoc commented 12 years ago

I've added a binary called diamond-setup that should work on installed machines to allow easier setup of diamond. Please test this and let me know of any issues that come up!

Samuel29 commented 12 years ago

Hi, I had a quick overview of this binary's code. I'm wondering who would need to enable all collectors by default ? on many of my servers I don't have all this services running simultaneously (memcached, mysql, puppet dashboard... ). IMHO the most logic setting would be to disable everything except "base" system metrics (cpu, network IO, etc).

Another remark : it would be great to set TAGS for major releases. I can't figure out which version is stable... ?

best, Samuel

kormoc commented 12 years ago

By default only the following collectors are enabled by default:

The rest are disabled by default. The Apcupsd one was a mistake.

I am planning to start tagging releases with the release of 3.0.0 in the next week or so. Just want to make sure all the code is nice and tight before I tag a stable release.

kormoc commented 12 years ago

@Samuel29 You can see from here: https://github.com/BrightcoveOS/Diamond/blob/master/src/diamond/collector.py#L98 That we do have the majority of collectors disabled. Are you seeing otherwise?