search

domainaware / parsedmarc

A Python package and CLI for parsing aggregate and forensic DMARC reports
https://domainaware.github.io/parsedmarc/
Apache License 2.0
1.02k stars 223 forks source link

JSON & CSV not updating when running via systemd #99

Closed aderrose closed 5 years ago

aderrose commented 5 years ago

Hi,

I have recently setup parsedmarc on my mail server, Debian 10 running MailCow.

I think everything is setup correctly as the DMARC reports get picked up and moved to the archive folder but the JSON output files only seem to update if I run from the command line.

I've configured a systemd unit as per the instructions which seems to run, move emails and at startup rebuilds the JSON file to a fresh (blank) state. However, the JSON files don't update as new messages come in.

I've checked the parsedmarc log which is empty and the syslog and systemd log for parsedmarc don't show any errors.

I'm obviously doing something wrong but I'm not sure what.

Any help would be appreciated.

Thanks.

seanthegeek commented 5 years ago

Hi,

When running as a systemd service, parsedmarc does not output files. Instead, you need to specify a config file with details for saving the data to an Elasticsearch or Splunk instance. That way, the data is parsed, saved and aggregated as soon as the reports come in.

You can export data from Elasticsearch or Splunk as JSON if you want.

aderrose commented 5 years ago

Hi,

I'm sending the JSON feed to a Web api for storage and display (when I build the UI) via a Cron job as I don't have the processing power to run Elasticseach or Splunk.

I don't process many DMARC records so this seemed the simpler solution but if I can't get JSON output via systemd I'll have to rethink.

Is it possible to run the app via Cron and have it close when completed?

seanthegeek commented 5 years ago

You could leverage the python API to do the parsing

Or

Like you said, you could run a cron job without specifying a parsedmarc config file, and it would dump the json to stdout

aderrose commented 5 years ago

Thanks for the tips, I'm going to be AFK for a couple of days but I'll look into options when I get back.

aderrose commented 5 years ago

I think I've got a working setup but will monitor for a few days before committing.

In a Bash script I'm firing parseDMARC with a config file as I need it to extract from an IMAP account, this will dump the JSON file correctly, if I leave out the "Watch" option from the IMAP config section the program will close after each run is complete. I've then put a cURL command to pass the JSON file to my API. These two commands are then triggered by Cron every hour.

Thanks again for your help.