dmarc reports every 5 minutes

[judouk]

I've managed to get a working setup and have parsedmarc processing emails. However, I'm getting flooded with parsedmarc report messages (via email) every 5 minutes and they are empty.

I've checked the inbox of the account I've setup and it is empty. The processed messages have been moved to the Archive folder as expected.

How can I setup parsedmarc to only send a summary email when there is something to report?

[RealSebFox]

I have a similar, or probably the same issue. ParseDMARC appears to restart every 5 minutes since I updated to version 8.0.2.

With debug logging enabled:

2022-04-27 00:28:50,092 - INFO - [cli.py:667] - Starting parsedmarc
2022-04-27 00:28:50,587 - DEBUG - [__init__.py:1072] - Found 0 messages in INBOX
2022-04-27 00:28:50,588 - DEBUG - [__init__.py:1080] - Processing 0 messages
2022-04-27 00:28:50,664 - DEBUG - [__init__.py:1395] - Emailing report to: me@home
2022-04-27 00:33:52,341 - INFO - [cli.py:667] - Starting parsedmarc
2022-04-27 00:33:52,828 - DEBUG - [__init__.py:1072] - Found 0 messages in INBOX
2022-04-27 00:33:52,829 - DEBUG - [__init__.py:1080] - Processing 0 messages
2022-04-27 00:33:52,904 - DEBUG - [__init__.py:1395] - Emailing report to: me@home

This happens precisely every 5 minutes.

[HansVanEijsden]

I have a similar, or probably the same issue. ParseDMARC appears to restart every 5 minutes since I updated to version 8.0.2. This happens precisely every 5 minutes.

Here too, but because in /etc/systemd/system/parsedmarc.service there's under [Service] the setting RestartSec=5m, according to https://domainaware.github.io/parsedmarc/#running-parsedmarc-as-a-systemd-service.

[RealSebFox]

I have a similar, or probably the same issue. ParseDMARC appears to restart every 5 minutes since I updated to version 8.0.2. This happens precisely every 5 minutes.

Here too, but because in /etc/systemd/system/parsedmarc.service there's under [Service] the setting RestartSec=5m.

Does that not mean it only restarts every 5 minutes, but not if it hasn't crashed before? If I'm correct here, I'd assume it crashes after booting, every 5 minutes and 5 minutes later it starts again.

Version 7.1.1 worked perfectly fine with the exact same service.

[HansVanEijsden]

Does that not mean it only restarts every 5 minutes, but not if it hasn't crashed before? If I'm correct here, I'd assume it crashes after booting, every 5 minutes and 5 minutes later it starts again.

That's a good one. I also have Restart=always listed. According to the systemd documentation you should be right: RestartSec=: If automatically restarting the service is enabled, this specifies the amount of time to wait before attempting to restart the service. Restart=: This indicates the circumstances under which systemd will attempt to automatically restart the service. This can be set to values like “always”, “on-success”, “on-failure”, “on-abnormal”, “on-abort”, or “on-watchdog”. These will trigger a restart according to the way that the service was stopped.

This means, as far as I can see, we both have the same issue: parsedmarc exiting. My debug logging shows the same as yours.

Here the output of service parsedmarc status:

     Loaded: loaded (/etc/systemd/system/parsedmarc.service; enabled; vendor preset: enabled)
     Active: activating (auto-restart) since Wed 2022-04-27 01:03:22 CEST; 1min 39s ago
       Docs: https://domainaware.github.io/parsedmarc/
    Process: 6665 ExecStart=/usr/local/bin/parsedmarc -c /etc/parsedmarc.ini (code=exited, status=0/SUCCESS)
   Main PID: 6665 (code=exited, status=0/SUCCESS)
        CPU: 1.599s

And journalctl -u parsedmarc.service -r:

Apr 27 00:48:15 vps systemd[1]: Started parsedmarc mailbox watcher.
Apr 27 00:48:15 vps systemd[1]: parsedmarc.service: Consumed 1.426s CPU time.
Apr 27 00:48:15 vps systemd[1]: Stopped parsedmarc mailbox watcher.
Apr 27 00:48:15 vps systemd[1]: parsedmarc.service: Scheduled restart job, restart counter is at 111.
Apr 27 00:43:15 vps systemd[1]: parsedmarc.service: Consumed 1.426s CPU time.
Apr 27 00:43:15 vps systemd[1]: parsedmarc.service: Deactivated successfully.
Apr 27 00:43:13 vps systemd[1]: Started parsedmarc mailbox watcher.
Apr 27 00:43:13 vps systemd[1]: parsedmarc.service: Consumed 1.399s CPU time.
Apr 27 00:43:13 vps systemd[1]: Stopped parsedmarc mailbox watcher.
Apr 27 00:43:13 vps systemd[1]: parsedmarc.service: Scheduled restart job, restart counter is at 110.

[RealSebFox]

I downgraded to 7.1.1. With pip3:

pip3 install --upgrade parsedmarc==7.1.1

And now I see this in my logfile:

2022-04-27 01:29:45,644 - DEBUG - [__init__.py:1115] - Found 0 messages in INBOX
2022-04-27 01:29:45,644 - DEBUG - [__init__.py:1123] - Processing 0 messages
2022-04-27 01:30:15,952 - DEBUG - [__init__.py:1115] - Found 0 messages in INBOX
2022-04-27 01:30:15,952 - DEBUG - [__init__.py:1123] - Processing 0 messages
2022-04-27 01:30:26,388 - DEBUG - [__init__.py:1115] - Found 1 messages in INBOX
2022-04-27 01:30:26,388 - DEBUG - [__init__.py:1123] - Processing 1 messages
2022-04-27 01:30:26,388 - DEBUG - [__init__.py:1127] - Processing message 1 of 1: UID 273209
2022-04-27 01:30:26,435 - INFO - [__init__.py:806] - Parsing mail from dmarc-support@wp.pl
2022-04-27 01:30:26,459 - DEBUG - [__init__.py:1180] - Moving aggregate report messages from INBOX to Archive/Aggregate
2022-04-27 01:30:26,460 - DEBUG - [__init__.py:1187] - Moving message 1 of 1: UID 273209
2022-04-27 01:30:26,705 - INFO - [elastic.py:298] - Saving aggregate report to Elasticsearch
2022-04-27 01:30:57,208 - DEBUG - [__init__.py:1115] - Found 0 messages in INBOX
2022-04-27 01:30:57,210 - DEBUG - [__init__.py:1123] - Processing 0 messages
2022-04-27 01:31:32,516 - DEBUG - [__init__.py:1115] - Found 0 messages in INBOX
2022-04-27 01:31:32,519 - DEBUG - [__init__.py:1123] - Processing 0 messages
2022-04-27 01:32:02,840 - DEBUG - [__init__.py:1115] - Found 0 messages in INBOX
2022-04-27 01:32:02,841 - DEBUG - [__init__.py:1123] - Processing 0 messages
2022-04-27 01:32:33,168 - DEBUG - [__init__.py:1115] - Found 0 messages in INBOX
2022-04-27 01:32:33,168 - DEBUG - [__init__.py:1123] - Processing 0 messages

That's how it's supposed to look.

[sableeyed]

Interesting.. Are both of you using IMAP to retrieve messages? I am using google oauth and my systemd unit never does anything, whereas both of yours continually crashes.

[HansVanEijsden]

Interesting.. Are both of you using IMAP to retrieve messages? I am using google oauth and my systemd unit never does anything, whereas both of yours continually crashes.

Yes @mlizbeth, I'm using IMAP to retrieve messages. On the same host, to local Dovecot. The login line from syslog:

Apr 27 17:15:08 vps dovecot: imap-login: Login: user=<dmarc@***.nl>, method=PLAIN, rip=2a02:2770:8:0:216:3eff***, lip=2a02:2770:8:0:216:3eff***, mpid=60699, TLS, session=<QBZdRqTdrjQqAidwAAgAAAIWPv/+rYyT>

[judouk]

Yes, I’m also using IMAP(S) to retrieve the emails I hadn’t noticed that it was crashing as I half expected I’d done something wrong when configuring it; I didn’t find the installation instructions to easy to follow and many of the guides sent me in the wrong direction

I haven’t tried downgrading it as others have mentioned as not had the opportunity today

[sableeyed]

I know it's not ideal, but you could change the watch directive to false, disable the systemd unit, and run parsedmarc on a cron job as an alternative to downgrading.

[RealSebFox]

I know it's not ideal, but you could change the watch directive to false, disable the systemd unit, and run parsedmarc on a cron job as an alternative to downgrading.

That is a good and valid suggestion, but it feels like a downgrade to me.

[sableeyed]

It is a workaround without having to downgrade the version. Just as downgrading the version is a workaround.

On Thu, Apr 28, 2022 at 1:23 PM RealSebFox @.***> wrote:

I know it's not ideal, but you could change the watch directive to false, disable the systemd unit, and run parsedmarc on a cron job as an alternative to downgrading.

That is a good and valid suggestion, but it feels like a downgrade to me.

— Reply to this email directly, view it on GitHub https://github.com/domainaware/parsedmarc/issues/309#issuecomment-1112526039, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAORR44VJ26NZTJ7KUZUFN3VHLJTLANCNFSM5ULYO7DQ . You are receiving this because you were mentioned.Message ID: @.***>

[RealSebFox]

It is a workaround without having to downgrade the version.

I prefer solutions over workarounds... Not just a little. A lot! ;-)

If a workaround is needed, downgrading is the fastest one.

[sableeyed]

Just trying to help out, I am not the developer ;-)

So far my systemd service has been running for 20 min no crashes, but thats with gmail API.. I will test on my own email server with IMAP this weekend and/or see if I can find the commit that broke it.

[RealSebFox]

Just trying to help out, I am not the developer ;-)

It's a good workaround. Thanks for sharing! I just happen to prefer version 7.1.1 for that matter.

[seanthegeek]

My apologies everyone. In my excitement to integrate native Microsoft 365 and Gmail support, I obviously didn't do enough testing of existing features before releasing 8.0.x.

I have yanked all of the 8.0.x releases from PyPI. All existing installs should downgrade to 7.1.1. I will release 8.1.0 when I am sure bugs introduced in 8.0 are fixed.

@nathanthorpe can you look into this and #311? They seem to be related to the refactoring done in #301. In the meantime, I'll do some testing. I need to rebuild the server that I was using for production, because apparently the VM was deleted..

[nathanthorpe]

Yeah I think I know what's up. Unfortunately I didn't get to testing the callback because I didn't have access to a server that supports IDLE...

[nathanthorpe]

The watch flag has been moved to the [mailbox] block. The program doesn't seem to be crashing, it is just closing down when it's done and then systemd is restarting it after 5 minutes.

A few options have been moved over to mailbox from the imap block to make it generalized to either MS, IMAP, or Gmail.

[seanthegeek]

Ah yes, that makes perfect sense. I've added a warning in logging output and documentation that will be included in 8.0.3, which will be released as soon as I have time to test and validate PR #313 as a fix for #311 (hopefully this weekend).

[seanthegeek]

8.0.3 is out, which fixes some issues and also adds a warning to move the watch option to the mailbox configuration section if it is in the imap configuration section.