Add log output handler plugins to bunyan cmd

[mhart]

Currently the bunyan cmd has a number of fixed output modes (OM_PAUL, OM_JSON, etc).

It would be great to be able to plug in custom handlers and filters (based on property matching/globbing or whatever). This is actually described briefly in this blog post:

records with a "metric" could feed to statsd, records with a "loggly: true" could feed to loggly.com

You can currently just patch the handleLogLine function to do this using your own custom logic, but it would be nice if a cleaner mechanism was supported out of the box.

[mhart]

Thinking about it a bit more - it may make more sense just to pipe the output through something that already does all of this, eg logstash - but I'll leave this issue open in case you guys do wanna consider it.

[trentm]

Currently the intention of the bunyan CLI command is to pretty-print, view (and eventually) filter bunyan format log files or streams.... not really as a tool to redirect those log records somewhere (like, say, to loggly).

That said, one thing that would be useful is to extract the code from the bunyan script into a "bunyan-reader" library that could be re-used by other node scripts to easily make a filter like you describe. That's currently on the TODO list: see "bunyan-reader" at https://github.com/trentm/node-bunyan/blob/master/TODO.md.

I'm not opposed to a plugin mechanism for the bunyan CLI (that would allow this).

[aseemk]

This issue seems to be about bunyan CLI, but is there any Loggly integration for the bunyan library? I see it has support for streams, but that's a bit different than making (ideally, batched) HTTP requests. I also see there's https://github.com/mcavage/node-bunyan-syslog and Loggly supports syslog, but syslog is plaintext, while the whole goal of bunyan is JSON logging. =) Thanks in advance!

[trentm]

I think node-bunyan-syslog is just logging the serialized JSON as the "plaintext". You can watch the syslog events (bunyan log records) on the other end and pretty-print with the bunyan CLI.

Loggly: a node-bunyan-loggly module would be appreciated. I don't use loggly so am unlikely to write one soon. A custom bunyan stream should make it straightforward to write to Loggly... including buffering bunyan records to make batched HTTP requests to loggly. This shows a quick example of a custom bunyan stream: https://github.com/trentm/node-bunyan-winston/blob/master/restify-winston.js#L18-L74 You could add batching to the .write() method there.

[simonexmachina]

Any further progress on this one? I'd also like to use Bunyan with Loggly.

[DaAwesomeP]

Well, someone's tried it: https://github.com/smebberson/bunyan-loggly

[simonexmachina]

Cool, thanks. I'll check it out.

[DaAwesomeP]

@trentm what is the "correct" way to add handler plugins? I'm looking into bunyan-logstash. However, I noticed that a similar library (bunyan-loggly) integrates itself into bunyan much differently. A real implementation of plugins for streams and documentation would be nice. Either way, between the two libraries, which integration with bunyan would you prefer? Basically, a require directly in the stream name/type or simply a new argument invoking a previously called on library?

[mhart]

Should I close this? Hasn't been any real updates on it in a while.

[simonexmachina]

I think so.