domainaware / parsedmarc

A Python package and CLI for parsing aggregate and forensic DMARC reports
https://domainaware.github.io/parsedmarc/
Apache License 2.0
986 stars 214 forks source link

Add support for Microsoft Graph API (Microsoft 365 mailboxes) #301

Closed nathanthorpe closed 2 years ago

nathanthorpe commented 2 years ago

Add support for fetching messages from Microsoft 365 mailboxes using the Graph API (issue #111).

Note that this requires an app registration with admin consent to the Mail.ReadWrite (delegated) permission.

Generalize the get_dmarc_reports_from_inbox and watch_mailbox to support different types of mailboxes with the MailboxConnection interface. This could be used for the getting messages from the Gmail API without having to write a separate get reports method.

kohtala commented 2 years ago

Hmm. This repository does not seem to use tags for versions, releases. Commit messages do not carry much useful info. I searched if semantic versioning is mentioned and looked at the CHANGELOG.md, but I did not see anything that would tell major and minor version numbers work by the semantic versioning rules. If semantic versioning is used, the changes as they are would require major number increase since it may break old installations unless configuration is modified.

I'm here because I commented on the issue about a change towards this, but I have not used parsedmarc yet. I use Google, so I can not take this into use with this alone. Thanks anyway, seems a step forward for me too.

Otherwise quickly skimming, I do not see other problems but the breaking change.

fluidum commented 2 years ago

Thank you for the effort of this!

I was able to build the PR, but I wasn't able to run it.

@nathanthorp can you refer what is missing here or share your working template of your configuration file?

# ./virtual/environment/bin/parsedmarc -c parsedmarc.ini
   ERROR:cli.py:598:You must supply input files, or an IMAP configuration

cat parsedmarc.ini

[general]
save_aggregate = True
save_forensic = True
offline = True
debug = True
n_procs = 1
chunk_size = 1

[elasticsearch]
hosts = https://<secret1>:9200/
ssl = True
cert_path = /etc/ssl/certs/ca-certificates.crt
monthly_indexes = True

[mailbox]
test = true

[msgraph]
client_id = <secret2>
client_secret = <secret3>
mailbox = <secret@secret.com>
nathanthorpe commented 2 years ago

@nathanthorp can you refer what is missing here or share your working template of your configuration file?

Hmm I have changed the error message on this line L639 of cli.py to You must supply input files, or a mailbox connection. From looking at your output, it seems to still have the older version of the code.

How did you build it on your end?

Also you will need to give it a username and password since it uses delegated permissions instead of application permissions. I could implement it using application permissions but then you'd need to grant it read/write access to all mailboxes, which is less secure.

seanthegeek commented 2 years ago

@nathanthorpe It looks like you used some syntax that requires Python >=3.8. Can you fix it to work with Python 3.6, so it will work with older Linux distributions?

https://github.com/domainaware/parsedmarc/runs/6119612789?check_suite_focus=true#step:7:27

nathanthorpe commented 2 years ago

Done, also let me know if more documentation is required for this. I only updated the main README but I noticed there was also one in docs/

nathanthorpe commented 2 years ago

I see that there was a PR related to Gmail, if you want I can try to merge it with the MailboxConnection interface to clean it up a little

seanthegeek commented 2 years ago

Sure. If you could also run the code through flake8 before submitting the next PR, that would be great. Thanks!

seanthegeek commented 2 years ago

Also let me know if more documentation is required for this. I only updated the main README but I noticed there was also one in docs/

For the config listing in the index in docs, I just copy/paste from the README.

seanthegeek commented 2 years ago

Actually, I just noticed that all of the pep8 errors are from the Gmail PR. I'll fix those.