jertel / dmarc2logstash

Injects POP3-polled DMARC feedback reports into Elasticsearch via Logstash and Filebeat.
MIT License
16 stars 4 forks source link

Parsing Error for legitim Dmarc XML Report #1

Closed Twibow closed 6 years ago

Twibow commented 6 years ago

Hello,

Thank's for your stuff, but I encounter two problem :

First : The script does not recognize attachments (with true DMARC Report). My mail is in "read" status in my mailbox, but dmarc2logstash consider it "Preserving email message since it is not a DMARC report;"

Second : I've test to send a XML DMARC report unzip directly in my dedicated mailbox. The script connect and read the mail, but don't interpreter it and generate a Python error : Traceback (most recent call last): File "dmarc2logstash.py", line 150, in <module> sys.exit(main()) File "dmarc2logstash.py", line 147, in main start(server, username, password, sleepSec, jsonOutputFile) File "dmarc2logstash.py", line 114, in start download(server, username, password, jsonOutputFile) File "dmarc2logstash.py", line 44, in download successCount = parseAttachments(jsonOutputFile, msg) File "dmarc2logstash.py", line 64, in parseAttachments if parse(jsonOutputFile, str(data)): File "dmarc2logstash.py", line 91, in parse record['auth_dkim_domain'] = child.find('auth_results').find('dkim').find('domain').text AttributeError: 'NoneType' object has no attribute 'find'

jertel commented 6 years ago

You'll need to provide a sample of one of these DMARC reports and emails. However, I don't see how it would be different. The DMARC reports I currently receive are all standardized, regardless of them being sent by Yahoo, Comcast, Amazon, etc. They are also consistently named, consistently gzipped, etc. So I can't help but wonder if you have some email preprocess that is manipulating the reports before they reach the dmarc2logstash POP3 account.

Twibow commented 6 years ago

Hello Jertel,

I'm comeback with news : I receive Yahoo Dmarc report this week and it passed correctly in your script !

So the issue seems to be in Google DMARC report only.

I send you an email with Google DMARC Report in attachement.

EDIT : Google send a DMARC Report in ZIP compression... erff

See you

jertel commented 6 years ago

Hello,

I pushed a new revision that should address both issues; 1) Google's choice to use ZIP instead of GZIP as is specified in the RFC, and 2) Missing XML elements. A new docker images is available with version 1.0.1.

Thanks for sharing the attachment that helped track down the problem.

Twibow commented 6 years ago

Hello Jertel,

Thank's a lot for your stuff !

1- You're right about the format (Zip) used by Google... Strange coming from them however.

2- Yes, It's missing DKIM in . I made contact with Google for know why. I will tell you more as soon as I have a feedback.