liuch / dmarc-srg

A php parser, viewer and summary report generator for incoming DMARC reports.
GNU General Public License v3.0
218 stars 32 forks source link

Suggested cron schedule? #63

Closed wwwguy closed 1 year ago

wwwguy commented 1 year ago

Hey brother... first of all, great job! Been looking for a solid DMARC parser for use with with a LAMP stack for a while. Good stuff!

I'm using the email option as the source for our reports. The utilities are running fine via terminal but before setting them up as crons I wanted to see if there is a recommendation for the settings? i.e. fetching reports 1 per day, mailbox cleaner (1 per day?), DB admin, reportlog/reports cleaner?

liuch commented 1 year ago

Hello! I don't have a general recommendation. I can only voice my thoughts on this subject.

fetch_reports

If you do not often look into the list of reports and if you receive no more than 10 incoming reports per day, then it will be enough to run fetch_reports 1-2 times a day. If you want to quickly monitor the reports you receive or you receive a lot of incoming reports, you should look at what time the reports from external mail servers arrive and adjust the cron job according to these observations. Any case, I do not see the need to retrieve reports from the mailbox more than once an hour. The main thing is not to allow the number of unprocessed incoming reports to grow in your mail box. Here is my cron job: 0 */6 * * * www-data cd /usr/local/share/dmarc-srg/www && php utils/fetch_reports.php > /dev/null.

mailbox_cleaner, reports_cleaner, reportlog_cleaner

In most cases, running these utilities once a day is sufficient. If the number of undeleted messages and records in the database is constantly growing, then you need to run the utilities more often.

summary_report

Here I think everything is quite obvious. Any way, you should not run this script next to the end of the report period, because some servers send their reports with some delay, that can last several hours and their reports will not be added to the report. It's mine: 0 11 * * 2 www-data cd /my/path/to/www && php utils/summary_report.php domain=mydomain.net period=lastweek

Some notes

It is desirable that the utilities do their work not at the same time. You may like also run the _cleaner utilites more often, but at the same time reduce the _maximum parameters in the conf.php file to spread the load over time. But this only makes sense if you have a lot of reports and you don't want to create peak load on your server.

I hope this helps.

wwwguy commented 1 year ago

Makes perfect sense (and actually, is pretty much exactly how we went ahead and set them). Really appreciate the response and detail though as it helps confirm our understanding of how they all fit together. Thanks again!