Puppet run reports in logstash

bobtfish commented 11 years ago

The puppet report plugin: https://github.com/youdevise/puppet-logstash-reporter

Web viewer for report info: https://github.com/youdevise/norman

combined, these make something with the same basic functionality for viewing puppet run reports as Foreman.

Todos:

Make the transport pluggable (lusis' original code used tcp, I've smashed in zeromq)
Views - all reports for one host.
Fix the test suite(s)

Dieterbe commented 11 years ago

so this would basically store events like "puppet ran on machine a/b and did change foo/bar" ? if you get that into elasticsearch i could work on showing the events on graphs with https://github.com/Dieterbe/timeserieswidget just let me know in time if you can make it work ;)

Dieterbe commented 11 years ago

hey i got the display on graphs working (see #59). so if you can show me how the puppet events look like within ES (i.e. which tags within event._source.@tags) i can easily tune the display accordingly

bobtfish commented 11 years ago

Current status for what I'm putting into ES is: http://paste.scsys.co.uk/237865 and http://paste.scsys.co.uk/237866

bobtfish commented 11 years ago

got a load of reports in my local ES server: http://10.10.22.91:9200/logstash-2013.03.29/puppet-apply/_search?pretty=1

bobtfish commented 11 years ago

I've also committed a script that allows you to re-index the test data I'm using into your own ES instance:

https://github.com/youdevise/norman/blob/master/scripts/reindex_runs.pl

Dieterbe commented 11 years ago

i'll be using your raw dump instead of reading from your computer directly

bobtfish commented 11 years ago

Pushed README to the norman repository showing how to actually set this stuff up :)

Dieterbe commented 11 years ago

current state. http://i.imgur.com/VqUhzxI.png edit: newer version http://i.imgur.com/w5cb9oz.png

obfuscurity commented 11 years ago

What are the actual commits and/or PR?

obfuscurity commented 11 years ago

I think this is what we're talking about. @bobtfish please confirm.

https://github.com/youdevise/puppet-logstash-reporter/compare/7ae74b0efc87206447c408445c6a77a1132f41ca...625cfee3e727e42ac3cea297b5c02cb2de17c2c7

obfuscurity commented 11 years ago

And...

https://github.com/youdevise/norman/compare/6dcb2d865d89583ff09353293e3a1ad053b87bac...df404f5735ddb330bfea6b41fae17a12bf3cd1c7#diff-168

bobtfish commented 11 years ago

No pull reqs yet, this stuff is standalone. (i.e. the web part will just be an ES plugin, and the puppet report will go on the forge)

Commits in the two repos linked in the issue description.

t0m

Jason Dixon notifications@github.com wrote:

What are the actual commits and/or PR?

— Reply to this email directly or view it on GitHub.

Dieterbe commented 11 years ago

the display on a graph part is https://github.com/Dieterbe/timeserieswidget/compare/master...es-wip

bobtfish commented 11 years ago

And yes - the diffs you picked are the stuff done today - I'd started a prototype on the plane over (being prepared!) :D

bobtfish commented 11 years ago

Yeah that + the other repos.

t0m

Jason Dixon notifications@github.com wrote:

I think this is what we're talking about. @bobtfish please confirm.

https://github.com/youdevise/puppet-logstash-reporter/compare/7ae74b0efc87206447c408445c6a77a1132f41ca...625cfee3e727e42ac3cea297b5c02cb2de17c2c7

— Reply to this email directly or view it on GitHub.

monitorama / hackathon

Puppet run reports in logstash #57