search

mailcow / mailcow-dockerized

mailcow: dockerized - 🐮 + 🐋 = 💕
https://mailcow.email
GNU General Public License v3.0
9.04k stars 1.18k forks source link

Feature Request: DMARC report parser #1341

Open normanu opened 6 years ago

normanu commented 6 years ago

It would be nice to have a parser automaticly readout the incoming reports mailbox and work it into a report like at

https://dmarcian.com/dmarc-xml/

andryyy commented 6 years ago

We could indeed add something like that. Not much effort. We would also need to add it to our DNS check.

We could use something like dmarc-reports@MAILCOW_HOSTNAME. A MX record does not need to exist, the A/AAAA record will be used as fallback. We can then parse it through a script and, it is a valid XML report, write it to our database.

normanu commented 6 years ago

I see http://dmarc.postmarkapp.com/ does it for free, but it might be better to keep it on the own server?

mkuron commented 6 years ago

http://dmarc.postmarkapp.com/ only supports a single domain per account. https://www.dmarcanalyzer.com also has a free tier, but it is quite limited. Building something into Mailcow would be nice, but doesn't need to be a super high priority -- one can deploy DMARC just fine without monitoring these reports.

ntimo commented 6 years ago

I just wanted to drop my opinion here too, I would love to have a dmarc analyzer build into mailcow. I currently use http://dmarc.postmarkapp.com/. But using a build in solution would be really great also in terms of privacy.

ledodev commented 6 years ago

Solved that for my personal setup with these: https://github.com/techsneeze/dmarcts-report-parser https://github.com/techsneeze/dmarcts-report-viewer Mailcow-official would be nice :)

ntimo commented 6 years ago

Would it be possible to implement this into Mailcow? https://github.com/techsneeze/dmarcts-report-parser https://github.com/techsneeze/dmarcts-report-viewer

stale[bot] commented 6 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

SomeGeek commented 6 years ago

Any updates on this @andryyy ? DMARC and other mail security features (like SRS) are increasingly important these days. It can be integrated individually, however, such important things should be part of the Mailcow core.

It somehow feels these things are not getting the priority it deserves.

andryyy commented 6 years ago

A DMARC parser will gain you security? There are a million services to parse them with nice reports etc. DMARC already is part of mailcow (Rspamd).

I don’t understand how a parser deserves higher priority. :-/ It’s just a nice to have, I guess.

SomeGeek commented 6 years ago

In my opinion DMARC doesn't make sense when you don't have insights in it. Sure, it can be done right now, but it should be a core feature.

andryyy commented 6 years ago

We already use DMARC, we just don't have an analyzer. This is not a core feature. It is a nice-to-have. Rspamd uses DMARC, it is not that we ignore it.

Clete2 commented 5 years ago

@SomeGeek @ntimo I have made a PR to integrate @ntimo 's request to add in the reporter.

https://github.com/mailcow/mailcow-dockerized/pull/2065

Adorfer commented 5 years ago

it would be very helpful to see progress here.

vain90 commented 5 years ago

https://hub.docker.com/r/gutmensch/dmarc-report Sounds pretty good.

develth commented 5 years ago

As i read from #2065 the dmarc report is welcome, but a docker container for that is overload. So it seems that the goal is to integrate it in the Interface (via PHP).

dragoangel commented 5 years ago

If somebody interested I created dedicated simple docker-compose for this stuff. It uses parsedmarc as parser and elasticsearch as storage and kibana as visualizer. On top it use nginx for basic auth and ssl. Maybe it will help somebody: https://github.com/dragoangel/parsedmarc-dockerized

borrelan commented 3 years ago

Adding 2cents.

In today’s email world, you need a proper feedback loop. I understand the fine line between must have and nice to have, but w/o a proper feedback loop it’s becoming harder and harder to make it i to a persons inbox w/o first being labeled as spam.

ThomDietrich commented 3 years ago

A few solutions were linked already. Did anyone configure a working configuration already?

While the question of a default component is not agreed, this could be documented in the third party tools area: https://mailcow.github.io/mailcow-dockerized-docs/third_party-borgmatic/

dragoangel commented 3 years ago

A few solutions were linked already. Did anyone configure a working configuration already?

While the question of a default component is not agreed, this could be documented in the third party tools area: https://mailcow.github.io/mailcow-dockerized-docs/third_party-borgmatic/

It already documenteddocumented at https://mailcow.github.io/mailcow-dockerized-docs/prerequisite-dns/

ThomDietrich commented 3 years ago

@dragoangel thanks for your answer. With "it" you refer to the usage of an external service. While I am generally not against that, for someone self-hosting a mailcow instance for the obvious reasons, this is not really the answer we are looking for.

Did anyone already include parsedmarc in their mailcow-dockerized setup? Would be nice if we could document that for broader adoption by others. Happy to contribute if no one steps forward.

dragoangel commented 3 years ago

Why you think should be part of mailcow stack?

I refer docs, which mentioned self hosted parsedmarc solution which can be freely deployed on same host as mailcow or dedicated one. This not should be a part of mailcow docker-compose project to work - it depends on heavy ELK stack to work with. There is a plenty people who use it and I'm part of them.

ThomDietrich commented 3 years ago

Let's not restart this discussion. The arguments were very clearly presented by half of the people involved in this thread.

It is totally fine if the vote is not to include this capability as a default (which I indeed agree with), but let's enable those who are interested by providing the right details to make this effortless. It's just silly that articles next to https://mailcow.github.io/mailcow-dockerized-docs/third_party-borgmatic give instructions for Nextcloud or Gitea, but not about a component crucial to have constant visibility on the DMARC integrity of your mail domain.

After a quick look it seems like there are two projects that offer the right stack to deploy parsedmarc:

The second one looks better to me. I might give it a try when it's not sunny outside ;)

lukaspavelka commented 2 years ago

@ThomDietrich do you have any news about this topic? PS: It is winter outside ;)

VermiumSifell commented 1 year ago

Would be nice if added @DerLinkman

ThomDietrich commented 1 year ago

I didn't get to it. Nor did anyone else in this thread it seems.

Thinking about it, this is the perfect proof for why a solution needs to be documented (or potentially offered). 15 participants and 5 years later, and people are still asking about it 🤷‍♂️

hanebuechenes commented 10 months ago

Any news here?

dragoangel commented 10 months ago

The point that it documented

https://docs.mailcow.email/getstarted/prerequisite-dns/?h=dmarc#optional-dmarc-statistics

But looks like people not like to read docs :P I don't think we should do this as part of mailcow. If people want it they can have it, as selfhosted or saas solution.

@DerLinkman I think we can close this issue honestly

borrelan commented 10 months ago

I read the docs. Maybe you could help stitch the docs with the platform?

All I see is external independent tools that are not integrated. Unless I missed something?

Update: In 2024 DMARC is more important than ever, yet the tools to manage it are not integrated to reflect its growing importance. I guess that the voice of the minority, but I have long moved away from self hosting as it's impractical without deliverability tools these days.

dragoangel commented 10 months ago

I read the docs. Maybe you could help stitch the docs with the platform?

All I see is external independent tools that are not integrated. Unless I missed something?

This dedicated topic that should not be "integrated", I don't understand why people want them to be part of mailcow. There 0 reason why they should be a part. Storing such historical data in ELK is heavy thing to add, it don't need on same host. Using SaaS at all not require you use your own server...

borrelan commented 10 months ago

You're not wrong, but the platform is useless without these tools to reach large email providers. I guess having your emails reach the destination is optional in your opinion?

ThomDietrich commented 10 months ago

Let's not close the issue. It is still very tangible and relevant. I've already summarized the current situation above. https://github.com/mailcow/mailcow-dockerized/issues/1341#issuecomment-932943594 Why are we having this discussion again 🙄

I admit, the biggest fault wrt this thread is that no one, including myself, did ever come around with a cookbook solution. I am personally not proud of this, but such is life. @borrelan @hanebuechenes I am convinced that the actual setup is quite simple, we just need one person to try, fail, improve, succeed, and document. Would you like to go for it?

This issue is resolved and can be closed as soon as an article on how to add "parsedmarc" to the mailcow stack was added to documentation.

borrelan commented 10 months ago

I'm not trying to stir the pot. Over the 20+ years of self hosting emails, I have given up the fight. The tools in this platform are awesome. In today's environment you need to know if your emails are reaching its destination. How you do that I will revert to those smarter than me.

When random emails consistently do not make it to its destination, you end up with a tool that no one trusts. In my experience, my users need confidence in the system in order to use it.

DMARC and the other DNS required entries are indeed a one time pain, but DMARC reports may provide insights that may help?

I can give it a go, but have moved on over 3/4yrs ago and sold out to the email cartels.