Inbound mail

[yunti]

I'm currently evaluating django-anymail (with mailgun) which I will be using inbound mail with and see on your roadmap that you intend to add inbound mail support. Did you have a structure/plan in mind for doing this, before I do a quick specific implementation similar to the mailgun docs, which probably won't be suitable for django-anymail?

[medmunds]

I've put inbound webhooks on hold until after the 1.0 release (see #4), because there seemed to be no interest in them. (So, thanks for expressing interest!)

If you're looking for something right away, I'd suggest just implementing your own Mailgun inbound webhook. It should be a pretty simple Django view. (You could borrow Anymail's Mailgun webhook validation code if that would help.)

By the way, you can use Anymail for outbound email while providing your own inbound email webhook. The two actually aren't all that connected.

The long-term plan for Anymail inbound handling will be very similar to sent-mail tracking: there will be a new Django signal for inbound events, and a new AnymailInboundEvent structure to normalize common inbound email parameters between ESPs (making it easier to switch or use multiple ESPs at once).

[yunti]

Thanks for the detailed response. Glad I mentioned interest, I think it will definitely help to make django-anymail the complete go to package to add in an ESP with django.

From an initial look I did feel that inbound and outbound were separate enough that I could use anymail + plus custom solution for inbound. But really appreciate you confirming that should work fine, as I'll probably go down that route for now as inbound looks to be a little bit off yet for d-anymail.

Think the structure you mentioned makes sense, this seems roughly similar to sendgrid-eventhooks which does a simple implementation of capturing the webhook in a view, parses the incoming email and emits a signal.

The other alternative out there I can see is django-inbound-mail but seems to use a different approach and convert the incoming HTTP back to an EmailMultiAlternatives object, which doesn't seem ideal particularly with the now more common use of using API to send.

Thanks for your comments - sorry for not offering to submit a patch - I'm probably just a bit too inexperienced at the moment.

[medmunds]

No problem. And your feedback is really helpful -- thanks.

[teolemon]

+1 for inbound. I want to migrate to django-anymail from djrill and get rid of mandrill at the same time.

[medmunds]

Re-opening by popular demand.

[yunti]

Congrats on v1.... just rementioning this as inbound would be a great addition. Thanks.

[pszpetkowski]

+1 I'd also find it very useful

[gertsteyn]

+1 would make ESP integration very clean

[medmunds]

An initial implementation of inbound handling for Mailgun is available in the feature_inbound branch.

I'd welcome some real-world testing and feedback on the API. Speak up if you're able to volunteer, and I'll bump whichever ESP you're using to the top of the implementation priorities.

Immediate next step is to get some initial docs in place. If you're itching to get started, inbound works a lot like tracking webhooks, but:

• The inbound webhook url to give your ESP is ".../inbound/" rather than ".../tracking/". [For Mailgun, you can also use ".../inbound_mime/", because MG changes behavior based on whether the url ends in mime.]
• The signal to listen to is called inbound rather than tracking (from anymail.signals import inbound)
• The event param for your signal receiver will be an AnymailInboundEvent, rather than an AnymailTrackingEvent. The inbound event has event_type (="inbound"), timestamp and event_id fields just like a tracking event. It also has a message property with all the good stuff.

The event.message property is an AnymailInboundMessage, which is a Python email.message.Message subclass, (hopefully) optimized for easy handling of inbound messages. [After some false starts, I settled on this because it can represent pretty much any inbound email message that might arrive. Django's EmailMessage is really designed for outbound, and is much more limited.]

AnymailInboundMessage has convenience accessors so you don't have to worry about navigating a MIME message structure or parsing headers. E.g., message.html is the HTML body of the message (if any). message.from_email.display_name and message.from_email.addr_spec are the parsed display name and email address of the From email header. message.attachments[0].as_file() is a Django UploadedFile object for the first attachment. And so on. But all the regular email.message.Message features are there, too, so you can access headers like message['DKIM-Signature'] or message.get_all('Received'). Until there are docs, the test cases have several examples.

TODO (roughly in order):

• [x] Basic inbound structure
• [x] Mailgun implementation and tests
• [x] Initial docs
• [x] Postmark
• [x] SendGrid
• [x] Mailjet
• [x] SparkPost
• [ ] ~Live tests?~ (not going to attempt this for now)
• [x] Mandrill (earlier if there are testing volunteers)
• [x] Finish docs, including ESP-specific info

[yunti]

Thanks for the initial implementation. I've recently started a new project, so could test this out hopefully in a few weeks. Mailgun is a good start as we are likely to use that. Thanks, I'll circle back in a few weeks.

[medmunds]

I've temporarily enabled RTD on the feature_inbound branch, and preliminary inbound docs are available at http://anymail.readthedocs.io/en/feature_inbound/inbound/. (The installation instructions have also been updated to cover setting up inbound webhooks. But no ESP-specific inbound info is documented yet.)

[medmunds]

The feature_inbound branch now has Mailgun, Postmark and SendGrid support.

Anyone wanting to test, you can use pip to install from that branch:

pip install -e git://github.com/anymail/django-anymail.git@feature_inbound#egg=django_anymail

(If you already have Anymail installed, you'll probably also need the --upgrade --no-deps pip flags.)

[medmunds]

Inbound support is in place for all supported ESPs, and the docs are updated too.

If anyone else is able to give it a spin, it'd be great to get some real-world testing in. See the previous comment for installation info.

(Also, if anyone's using Mandrill, it's not entirely clear from their docs whether their webhook authorization scheme applies to inbound events. I've assumed yes, but would be great to verify.)

[medmunds]

Inbound has been released in v1.3. Feedback still very welcome (just open new issues), as I don't think it's received much real-world usage yet.