liuch / dmarc-srg

A php parser, viewer and summary report generator for incoming DMARC reports.
GNU General Public License v3.0
223 stars 33 forks source link

Fetch all e-mail reports #95

Open KlaasBonnema opened 1 year ago

KlaasBonnema commented 1 year ago

The current IMAP implementation will only fetch UNSEEN e-mails and move processed mails to an alternate folder and/or mark them as SEEN. This will give problems if another process (or human user) is also looking at the raw mails. Allow the selection of ALL e-mails from a folder. We can already prevent moving processed mails to another folder with a config setting. This will cause mails to be selected with every fetch run but they will be skipped if processed before. Considering you will probably run the fetch once a day this would be acceptable.

liuch commented 1 year ago

This will cause mails to be selected with every fetch run but they will be skipped if processed before. Considering you will probably run the fetch once a day this would be acceptable.

This ceases to be acceptable when the daily volume of reports is large enough and there is a limit on the number of reports that can be processed at a time.

This utility was originally written with the assumption that no one looks into the mailbox but it. And if someone does, it is only to control the work of the utility. The obvious solution to this problem is to set up redirecting copies of the reports to the responsible person's mailbox. It might make sense to add such an option with a risk warning. I'll think about that.

williamdes commented 1 year ago

I have one workflow affected by this, when I select the first email of the list on Thunderbird it makes it read, I have to put in unread and finish my selection before doing a right click and copy to my DMARC folder Basically I manually copy emails from some clients into the ingestion folder.

liuch commented 1 year ago

When I wrote about copying reports, I meant specifying something like a server-side Sieve rule or one more mailbox in the DNS record _dmarc.yourdomain.net

KlaasBonnema commented 1 year ago

I receive the raw dmarc report mails with pop3 in a Thunderbird account. This way I build an archive of all raw dmarc report mails. A new dmarc report will however show as a new incomming message, distracting me from other incomming messages so I set a message filter to mark the new dmarc reports as read.

I made a change in dmarc srg to replace UNSEEN with ALL to process all dmarc emails, even if marked as seen and I set the max number of reports to process to unlimited. I can control the number of e-mails remaining in the mailbox with a Thunderbird setting and thereby control the overhead incurred.

If you allow to configure this with a proper warning text regarding limiting the maximum number of e-mails then that would make me happy.