Feature: TLS-RPT Support in the Viewer

[jnew-gh]

I've been working on an enhancement to dmarcts-report-parser to add TLS reporting (see Issue 65). Since any addition to the parser would be useless without a corresponding addition to the viewer, I have started modifying the viewer to incorporate TLS reports.

As a proof-of-concept, I simply copied the dmarcts.php files to tlsts.php files and quickly modified them to display the TLS reports. I have managed to get most of the features working in TLS reports (JSON highlighting is eluding me at the moment).

What I have found is that the main files (dmarcts- and tlsts-report-viewer.php) and the -list.php files are similar enough that I can merge them and deal with the differences in one set of files (say, viewer.php and viewer-list.php). I haven't yet made a decision about the -data files; they are different enough to maybe warrant separate files (say, tls-viewer-data.php and dmarc-viewer-data.php). I will have to do some more investigation.

Because this feature introduces major new functionality, I think it warrants a version bump. @techsneeze, could you please create a v2.0 branch or something similar? Also, maybe label this issue "enhancement"?

[jnew-gh]

I've made a pull request to add TLS-RPT functionality in the v2.0 branch of the viewer. I've also made a pull request to the parser that populates the database with the TLS-RPT data.

This code should be considered alpha quality not suitable for use in a production environment. I have done some basic testing and everything seems to work except the following:

• TLS Report options (partially implemented)
• Save current settings as initial view (partially implemented)
• JSON highlighter/syntax highlighter (not yet implemented)
• Status icons (the gree/red/orange/yellow circles beside the entries in the report list) (uncertain implementation) Working as of 2022-08-04

For some reason, I can't get SMTP TLS Reporting to work with my domains so all I get are failure reports. As a result, I don't really know what a successful TLS-RPT looks like. If someone wants to post some here so I can test them, it would be appreciated.

@ekalin, could you please review my PostgreSQL code if you are able. I have tried to duplicate your PostgreSQL additions as closely as possible but I don't have a PostgreSQL server on which to test.

[ekalin]

Here's the contents of a valid report:

{"organization-name":"Google Inc.","date-range":{"start-datetime":"2022-07-25T00:00:00Z","end-datetime":"2022-07-25T23:59:59Z"},"contact-info":"smtp-tls-reporting@google.com","report-id":"2022-07-25T00:00:00Z_kalinowski.com.br","policies":[{"policy":{"policy-type":"sts","policy-string":["version: STSv1","mode: enforce","mx: mail.kalinowski.com.br","max_age: 86401"],"policy-domain":"kalinowski.com.br","mx-host":["mail.kalinowski.com.br"]},"summary":{"total-successful-session-count":2,"total-failure-session-count":0}}]}

I'll try to get an actual email message with the attached report.

[jnew-gh]

Thanks for the report.

So I guess I did get TLS-RPT to work because that's the same type of report that I get, although I get failure details as well.

I'll work on getting the icons to show up properly.

[ekalin]

And here's a file with a report, in case it helps. tlsrpt.zip

And now I need a report with failures, to test the queries against the second table.

[jnew-gh]

And here's a file with a report, in case it helps. tlsrpt.zip

Thanks!

And now I need a report with failures, to test the queries against the second table.

Ooh, I have lots of those :-) {"organization-name": "Microsoft Corporation","date-range": {"start-datetime": "2022-06-16T00:00:00Z","end-datetime": "2022-06-16T23:59:59Z"},"contact-info": "tlsrpt-noreply@microsoft.com","report-id": "132999757500546328+example.com","policies": [{"policy": {"policy-type": "sts","policy-string": ["version: STSv1","mode: testing","mx: mail.example.com","mx: mail.example.org","max_age: 86400"],"policy-domain": "example.com"},"summary": {"total-successful-session-count": 0,"total-failure-session-count": 2},"failure-details": [{"result-type": "certificate-host-mismatch","receiving-mx-hostname": "example.com","receiving-ip": "100.100.100.246","failed-session-count": 1},{"result-type": "validation-failure","failed-session-count": 1,"failure-reason-code": "MX_HOSTS_MISMATCH"}]}]} (code redacted to generic domains)

[ekalin]

Insertion and retrieval from the Postres db worked fine, so it seems the database issues are resolved.

[jnew-gh]

Thanks for reviewing that code!

[Maeglin73]

Starting to test the viewer changes today and so far, so good. Thanks for the work you put into this!