Plugin for monthly statistic reports

[adestis-mb]

Hello Haraka Community,

today I just stumbled over the Haraka project when I was looking for an alternative to Postfix. One thing I could not found was an existing solution for regular statistic reports.

How can they be achieved? Do I have to write an own module for it?

Regards Markus

[smfreegard]

Hi Marcus,

There's no plugins that do statistical reports currently, so you'd have to write one. It all depends on what you want to report and how often as to how difficult that might be.

Kind regards, Steve.

[msimerson]

How can they be achieved?

As Steve said, it all depends on what you want to measure. Haraka spits out volumes of logs. Basic reporting can be achieved with some well crafted grep commands.

The PR #906 contains a plugin for logging to Elasticsearch. By storing traffic data there, it gets really easy to generate saved queries useful for reporting, pretty graphs of the data (with Kibana), and a dashboard.

[adestis-mb]

Too bad. Thanks for the suggestion (grep) but before I start with grep I really would write a plugin which hooks the necessary events and create statistic files (e.g. json format) which can then be post processed to something good looking.

I had a look at a lot of the plugins in the past hours and it seems to be more reasonable to create a statistics plugin. If our internal decision will be Hakara we will create a few plugins and publish them here..

Regards

[msimerson]

I really would write a plugin which hooks the necessary events and create statistic files (e.g. json format)

Do you realize that the format of documents getting inserted to Elasticsearch is JSON? Grab that plugin, remove the ES specific bits, and write the JSON files to wherever you want.

[smfreegard]

Markus - maybe if you gave us an idea of exactly what you want to achieve, we could probably give you a better answer.

The ES stuff is great; but if you only want some basic daily/weekly/monthly reports by domain and globally, then it could easily be like using a sledgehammer to crack a nut.

Depending on what you want you could easily do some top-level stuff using Redis (which is what I do here) and then dump that into JSON files once per day etc. If you want to graph stuff or do anything with timeseries data, then take a look at the redis-timeseries npm module which is what I use here.

[abhas]

Sometime back I wrote a small mail counter plugin to do something similar. The plugin stores logs in MongoDB. I then wrote another sinatra app to download the logs in the CSV format from MongoDB.

You can check out the code on this gist: https://gist.github.com/abhas/f240834f59eebe3b636a Its not very elegant but it works. You will need to install the winston nodejs module to use this.

[adestis-mb]

Thanks a lot for the good suggestions. I will have a look at the plugins. I would say the approach to store the data in database and have it aggregated later would be the best for us. I would then use the http protocol (REST) to request the statistics. Does anybody of you know the status of the http integration? I don't want to mess up thinks (new featues) which are currently under development.

[msimerson]

Does anybody of you know the status of the http integration?

It's merged, and if you're familiar with Express, its straight forward to add routes to it.