record_outgoung_message fails if the message contains attachments with UTF-8 filenames

[alex-voodoo]

The case is simple: send an email that has an attachment that has Cyrillic filename, then try to feed it into the mailbox. Here is a snippet:

from django.core.mail import EmailMessage

def send(request):
    outgoing_message = EmailMessage('subject', 'body', 'from@gmail.com', ['to@gmail.com'])
    uploaded_file = request.FILES['attachment']
    outgoing_message.attach(uploaded_file.name, uploaded_file.read(), uploaded_file.content_type)
    outgoing_message.send()
    sent_message = mailbox.record_outgoing_message(outgoing_message.message())

The outgoung message is delivered to the destination address; here are headers of an attachment in one of such test emails that I sent:

Content-Type: image/gif MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename*="utf-8''%D0%A2%D0%B0%D0%BD%D0%BA%D0%B8.gif"

However the django mailbox fails to record the message. The traceback is as follows:

File "/usr/lib/python2.7/dist-packages/django/core/handlers/exception.py", line 39, in inner response = get_response(request) File "/usr/lib/python2.7/dist-packages/django/core/handlers/base.py", line 249, in _legacy_get_response response = self._get_response(request) File "/usr/lib/python2.7/dist-packages/django/core/handlers/base.py", line 187, in _get_response response = self.process_exception_by_middleware(e, request) File "/usr/lib/python2.7/dist-packages/django/core/handlers/base.py", line 185, in _get_response response = wrapped_callback(request, *callback_args, *callback_kwargs) File "/usr/lib/python2.7/dist-packages/django/contrib/auth/decorators.py", line 23, in _wrapped_view return view_func(request, args, **kwargs) File "/projects/sample/app/views.py", line 100, in send sent_message = mailbox.record_outgoing_message(outgoing_message.message()) File "/usr/local/lib/python2.7/dist-packages/django_mailbox/models.py", line 226, in record_outgoing_message msg = self._process_message(message) File "/usr/local/lib/python2.7/dist-packages/django_mailbox/models.py", line 354, in _process_message message = self._get_dehydrated_message(message, msg) File "/usr/local/lib/python2.7/dist-packages/django_mailbox/models.py", line 240, in _get_dehydrated_message self._get_dehydrated_message(part, record) File "/usr/local/lib/python2.7/dist-packages/django_mailbox/models.py", line 264, in _get_dehydrated_message raw_filename = msg.get_filename() File "/usr/lib/python2.7/email/message.py", line 687, in get_filename return utils.collapse_rfc2231_value(filename).strip() File "/usr/lib/python2.7/email/utils.py", line 318, in collapse_rfc2231_value return unicode(rawval, charset, errors) TypeError: decoding Unicode is not supported

I tried to encode the filename of the attachment when adding it to the message, but that does not work at all: such message fails to be sent. It seems to me that using the original filename 'as is' is the right way.

[coddingtonbear]

Hey there, @alex-voodoo -- thank you for taking the time to post a bug report! I'd be glad to have a look, but first, could you post a sample e-mail message?

[alex-voodoo]

Hey @coddingtonbear, I'd be happy to do so, but how? This issue only affects outgoing messages, the incoming ones are accepted just fine and stored in the DB with all Cyrillic-named attachments.

Anyway, I have attached what GMail shows as the original message—that is exactly the one that record_outgoung_message has failed on. I have obfuscated the to and from addresses in the file as well as my local IP, all the rest is untouched.

If that doesn't help, please see again my explanation above of how to reproduce the problem. I believe that all you need is to attach a file with non-Latin filename. original_msg.txt

[ad-m]

@coddingtonbear , it looks attached message was parsed properly. https://github.com/coddingtonbear/django-mailbox/compare/cyrilic-attachment

[ad-m]

This same errors was received after:

    def test_message_outgoing_cyrilic_constructed(self):
        msg = EmailMessage('subject', 'body', 'from@gmail.com', ['to@gmail.com'])

        msg.attach('\xd0\xa2\xd0\xb0\xd0\xbd\xd0\xba\xd0\xb8.gif'.decode('utf-8'),
                   self._get_email_as_text('message_with_cyrillic_attachment.eml'),
                   'image/gif')
        self.mailbox.record_outgoing_message(msg.message())

        self.assertEqual(msg.attachments.all().count(), 1)

[coddingtonbear]

I spent a half hour or so trying to understand what's going awry on this one, but I ran out of time for the moment. I'm curious to hear if you had any ideas, @ad-m?