Maybe Update?

[dwt]

Hi there,

thank you so much for your blog post, it helped me a lot getting my own dmarc visualiser up and running. I've actually created a fork here https://github.com/dwt/dmarc-visualizer where I've simplified the docker file to build less local containers and use bind-mounts instead to get config files into the containers. I've also added a makefile to drive configuration and especially allow use with podman-compose to easily deploy it rootless.

What do you think? I'd love to get this into a pull-request or maybe get the guy from parsedmarc to add this to his documentation.

Best Regards, Martin Häcker

[zozs]

Hi!

Sorry for the silence, I just came back from a vacation, but your fork sounds really promising! I'll take a look at it as soon as I have the time, and see if I can merge it :slightly_smiling_face:

Best regards, Linus

[dwt]

I'd love to hear what you think - for sure there is still some extraneous code in there, but it is what I have currently deployed and am working on as I'm working on my deployment.

For sure it can be refined further. :-)

[ClearPathDigital]

@dwt and @zozs How do I connect either of these projects to an IMAP or POP account containing the reports? Can we get some documentation?

@dwt I am unable to comment on your fork of this because you don't have Issues enabled.

Thanks!

[dwt]

@ClearPathDigital: My fork has specific config variables for the IMAP settings and will auto-generate a config file from that. Have a look at the code!

I've not yet enabled issues, because I was hoping to only have a short lived repository. Lets see if I change that.

[ClearPathDigital]

@dwt, yes I have already done exactly that: I looked at the code. I put the config variables in place using the text files listed in the code. The parsedmarc process took a while to run. It seems to have accessed my mailbox with the reports in it, because they've all been marked as read. But, ultimately I got an error back and no data in grafana: ERROR:cli.py:605:IMAP Error: 285

[dwt]

@dwt https://github.com/dwt, yes I have already done exactly that: I looked at the code. I put the config variables in place using the text files listed in the code. The parsedmarc process took a while to run. It seems to have accessed my mailbox with the reports in it, because they've all been marked as read. But, ultimately I got an error back and no data in grafana: ERROR:cli.py:605:IMAP Error: 285

Hm, the setup is what I’m running, so I’m a bit lost. If the reports are read, accessing the mailbox should be fine?

You probably have to debug this down to what that error actually means. Sorry mate.

[zozs]

Hi @dwt !

I've taken a look at your repo, and I think you have made some great enhancements!

A few thoughts:

• Do you think it would be possible to have the SMTP part optional? If I understand it correctly, it only sends the aggregated report somewhere, which I don't think is always desired if you just want to look at the results in Grafana instead.
• Would it be possible to simplify trying it out a bit? In the original repo, you can just drop a "test" report in the files/ directory, and then spin the containers up to try it out. In your fork the user would have to both:
  1. Register a Maxmind account to get the GeoIP database,
  2. Find e-mail credentials for an account with the dmarc reports (and in the case of Gmail, generate an application password)
• which makes it a bit more "annoying" to just try it out :). Would it be possible to make it easier to get started, e.g., by making GeoIP optional and/or supporting files for testing?

I'd be happy to merge a PR after discussing the above a bit :)

[dwt]

Hi @zozs,

• SMTP: Definitely. I would like to change the makefile so that the values are used when present and are ignored when not. Not quite sure what the easiest way to do this would be. Do you have an idea for that?
• maxmind: Yes, I had that lying around anyway, so I added it in. Ideally the makefile would check if the secret file exists and condition on that. I somehow think that the makefile input file dependency should be usable for that, but I haven't figured out how to make make play along.
• credentials for smtp/imap: Being able to try it out from a folder is definitely something that should be retained. Ideally, I would like to have a report directory with sample reports as part of the repository. Do you happen to have something lying around that could be used? (I only have stuff that is confidential that I cannot easily share).

PRs would be more than welcome - what would be the easiest way to work on this? If we work with branches, I'd love to give out push access to my repo to prepare this? We could also work on branches on this repo? What would you prefer?

[dwt]

@zozs ping? I'd love have some progress on this, ideally with some input from you. So, could you spend some time to answer some of my questions?

[zozs]

@dwt sorry for the slow answer! I've had limited time to spend on this project the last month unfortunately.

• SMTP: Hmm, yeah it is a bit tricky. On the other hand, your PR already contains a template.py script, so I guess we could just make that script a bit smarter and simply disregard the whole [smtp] section if the environmental variables are empty/unset. We could also just test and see how parsedmarc reacts to empty strings in those variables, perhaps it just refrains from sending e-mails then too, and then we could just leave the variables empty.

• maxmind: Hmm, yeah, I'm not sure how to describe such behaviour in the makefile itself. Maybe we can just condition it using regular shell constructs like if [ -f secrets/maxmind_account_id.txt ] and run commands only then?

• sample reports: hmm, I don't think I have any samples I can make public either, unfortunately.

With regard to the work flow, we could work in a branch in your repo, and then when finished, we could just create a PR and merge it back into this repo. I think that would be a reasonable solution?

[dwt]

I'll get you invited to collaborate on my repo

[jsalatiel]

Any progress on this ?

[dwt]

@jsalatiel sadly not. To be honest, this completely fell through the cracks for me. :-(

@zozs: I did invite you as a collaborator. How can we get this best restarted? What would you like us to complete first to slowly get a merge started.