domainaware / parsedmarc

A Python package and CLI for parsing aggregate and forensic DMARC reports
https://domainaware.github.io/parsedmarc/
Apache License 2.0
960 stars 209 forks source link

Add support for analysing SMTP TLS reports #71

Open ghost opened 5 years ago

ghost commented 5 years ago

Hi

Have you planed that your tool can analyze the report from mta-sts (TLSRPTv1) too?

Or do you know another software for this?

Thank you for help

freddieleeman commented 5 years ago

URIports supports MTA-STS and DANE TLS-RPT reports.

seanthegeek commented 5 years ago

I just glanced over the RFC. This looks like it would be easy to add. Not sure when I'll get to it though.

dragoangel commented 4 years ago

@freddieleeman all point in self-hosted solutions is that they self-hosted. You not share your personal and company info with 3rd parties when you have possibility host own solution. It will be cool if parsedmarc will have this future :+1:

dragoangel commented 4 years ago

Hi @seanthegeek I can send you sample data for mta-sts and tlsa reports if it will help you. Do you need them?

seanthegeek commented 4 years ago

@dragoangel That would be great!

dragoangel commented 4 years ago

@seanthegeek I contact you in PM on twitter

NoSubstitute commented 4 years ago

@dragoangel That would be great!

Do you need more reports? I'm getting some incoming, and no clue how to read them. :-)

seanthegeek commented 4 years ago

I haven't even had the chance to read over what @dragoangel sent me. 😋 I'll keep that in mind though. Thanks.

dragoangel commented 4 years ago

@NoSubstitute no clue how to read them

TLS Reports are JSON formated mostly to one line. To get there more visibility you need format them to pretty json (using online tools, or text app, e.g: Notepad++ JSTool plugin, etc).

NoSubstitute commented 4 years ago

@NoSubstitute no clue how to read them

TLS Reports are JSON formated mostly to one line. To get there more visibility you need format them to pretty json (using online tools, or text app, e.g: Notepad++ JSTool plugin, etc).

Thanks. The NP++ plugin made it easier to read. Still, a nicely aggregated statistical view would be nice.

rhclayto commented 3 years ago

It would be a splendid feature. Is anyone aware of a currently existing self-hosted TLS-RPT analyzer/dashboard?

tbsmark86 commented 2 years ago

I've done a quick solution for this writing only to json output; so no analyzing or anything just re-using the automatic imap-mailbox handling. (It's enough for my use case)

Maybe some on can use this as a starting point for a real implementation: https://github.com/tbsmark86/parsedmarc/commit/990f8df60baa017fcd5aad9871c00f70f3dcafc7

EmadFathy commented 2 years ago

+1 for TLS reports feature.

mapietru commented 2 years ago

+1 for that feature.

PHPGangsta commented 2 years ago

+1, please support it

bbccdd commented 1 year ago

+1 !

cbrandlehner commented 1 year ago

+1 for MTA-STS support

ArenTahmasian commented 1 year ago

+1 for MTA-STS

matthiasmaes commented 1 year ago

+1 for MTA-STS

zell-mbc commented 1 year ago

+1 for MTA-STS

Zoey2936 commented 1 year ago

Is there something new on this? I've discovered this project today and for my use case is this is the only feature, which I miss.

mkilijanek commented 1 year ago

+1 !

ermurenz commented 1 year ago

+1 for MTA-STS support

Pascal76 commented 1 year ago

+1 :)

Kuzuto commented 11 months ago

To those who want TLS Reporting.. Have you set it up, and receiving reports ? Can I get some replies from who you receive TLS Reports from ?

I can start out:

freddieleeman commented 11 months ago

And an additional 20 that lack significance.

cbrandlehner commented 11 months ago

I agree that google.com and microsoft.com are majority of emails here.

However, adding the following:

freddieleeman commented 11 months ago

Are you sure? Haven't seen a single report, and we process thousands daily.

If they do, they are probably not RFC compliant.

mkilijanek commented 11 months ago

SMTP-TLS reports I receive are from Google.

Kuzuto commented 11 months ago
  • Google Inc.

    • Microsoft Corporation

    • SocketLabs

    • Comcast

    • Mail.ru

    • Mimecast

And an additional 20 that lack significance.

It'a amazing so many sends TLS Reports now. Only a few years ago, only 4 was sending reports, where Microsoft was the last coming to the group. : https://www.mailhardener.com/blog/microsoft-has-begun-sending-smtp-tls-reports Never got any from Mimecast. Would like to see that report. Is the TLS reports you receive from all those different senders RFC Compliant, or is any still lacking behind ?

jeffrysleddens commented 11 months ago

In order of magnitude:

I have almost 2000 SMTP TLS reports saved up.

zell-mbc commented 11 months ago

Small business email server:

wblondel commented 9 months ago

I receive TLS reports from:

Ressy66 commented 8 months ago

TLS-RPT will never take off because nowhere can I find information on how to generate and SEND reports, nobody else I knows, knows either, so its only half baked idea with results if you get mail from google and microsoft, nothing from the 50 million other mail servers out there

dragoangel commented 8 months ago

TLS-RPT will never take off because nowhere can I find information on how to generate and SEND reports, nobody else I knows, knows either, so its only half baked idea with results if you get mail from google and microsoft, nothing from the 50 million other mail servers out there

https://datatracker.ietf.org/doc/html/rfc8460 it has own rfc thing, how you can say after that it's half baked? If you like details - read rfc :) . Fact that there no much support (only ~10 big providers), yes, but not 2. It's because there no open source software that support parsing it or sending it.

Kuzuto commented 8 months ago

TLS-RPT will never take off because nowhere can I find information on how to generate and SEND reports, nobody else I knows, knows either, so its only half baked idea with results if you get mail from google and microsoft, nothing from the 50 million other mail servers out there

I kind of agree with you. TLS-RPT is easy to implement for the domain owner, but to get the full use of it, your mail service provider need to support MTA-STS or DANE as well. And this is something many mail service providers don't even support yet! Hence the reason we don't get reports.. Not everybody is only sending and receiving messages between Google and Microsoft.. Both the sending and receiving part, needs to support MTA-STS to prevent message from getting delivered, even when TLS fails. And when TLS fails .. well.. you get a TLS-RPT within the next 24 hours telling you, "Google was unable to deliver messages to you, because TLS was failing". No company will accept to get this information up to 23:59 hours later. So what is the point ? Many mail service providers is able to define inbound TLS rules and versions, to ensure TLS delivery. It's even possible to do exceptions.. That is not possible with MTA-STS.

Just like BIMI.. sounds good on paper, but all the big players have removed their BIMI records. It is expensive to buy certificate, and when only a few mail clients support it, and required by the MTA servers .. it dies out.

Microsoft support MTA-STS (last year) https://techcommunity.microsoft.com/t5/exchange-team-blog/introducing-mta-sts-for-exchange-online/ba-p/3106386

NOTE: Messages will be delivered when only one party supports MTA-STS. For example, when an MTA-STS-protected domain receives a message from a sender domain that doesn’t support MTA-STS, the message is delivered. The message is also delivered when the recipient domain doesn’t support MTA-STS, but the sender domain does. The only scenario where messages aren’t delivered is when both sides are using MTA-STS and MTA-STS validation fails.

Again from : https://www.mailhardener.com/kb/smtp-tls-reporting

So, the SMTP session always starts in plain text, and only switches to encrypted communication after the STARTTLS command is issued. Because TLS can fail in more ways than plain text connections, most MTAs will fall back to plain text if for some reason the TLS connection does not work.

With the introduction of [MTA-STS], it is now possible to enforce the use of TLS when receiving email. The plain-text fallback will no longer be allowed, so if TLS fails the email will not be delivered and returned to its sender.

I only see a change of TLS-RPT use, when more MTA servers support MTA-STS or DANE. And they need to support both.. because both is possible to enforce TLS connections. Else, what is the report good for? Please enlighten me, if TLS-RPT is a company critical thing.

Edit: this one is funny (start from bottom).. DANE not ready yet.. after 4 years https://techcommunity.microsoft.com/t5/exchange-team-blog/support-of-dane-and-dnssec-in-office-365-exchange-online/ba-p/1275494

seanthegeek commented 6 months ago

I've finally started working on this, as you can see in PR #453. Lots of work still needs to be done, so I would appreciate any help!

ermurenz commented 5 months ago

Most of SMTP-TLS reports we receive are from Google.com and microsoft.com.Some other more from mail.ru.