adriancdperu
commented
11 years ago Dear Tagomoris san Sorry for opening this, it seems this plugin https://github.com/sonots/fluent-plugin-grepcounter does the job, there wouldnt be any need to change flowcounter
Dear Tagomoris san
When Flowcounter does its job, it produces a log like:
2013-07-19 12:25:44 +0900 flowcounter: {"count":0,"bytes":0,"count_rate":0.0,"bytes_rate":0.0}
Which is nice because we can use this log as input of fluent-plugin-notifier or fluent-plugin-mail to send notifications.
Imagine Flowcounter is measuring number of error logs per minute. Then, it sends a log per minute and if threshold is above 10, fluent-plugin-notifier sends a warning. This warning is read by fluent-plugin-mail and a "too many errors" mail is sent. The problem is that in such mail we cannot see the error logs contents. fluent-plugin-mail has a "out_keys" feature to send log content, and fluent-plugin-notifier also saves the original log (if i remember well, in a "value" place). But if I'm correct, flowcounter plugin does not have feature to show original log content inside flowcounter log. Maybe it would be nice to make an option in flowcounter to add the original log contents to flowcounter's output?
Something like this
2013-07-19 12:25:44 +0900 flowcounter: {"count":0,"bytes":0,"count_rate":0.0,"bytes_rate":0.0, "content": "all the logs that flowcounter counted in 1 minute"}
This is maybe helpful if flowcounter is counting "error logs", but I know flowcounter main function is to count not to store or copy logs, so I understand why you did not add this feature.
I'm trying to add this option for my personal use but I don't know Ruby so I am having trouble (https://github.com/adriancdperu/fluent-plugin-flowcounter/blob/master/lib/fluent/plugin/out_flowcounter.rb) any insights will be greatly appreciated.
Adrian