search

python / cpython

The Python programming language
https://www.python.org/
Other
61.22k stars 29.53k forks source link

Email part with content type message is multipart. #89789

Open 5d819bd6-fcec-41bf-8190-b47d6c454d81 opened 2 years ago

5d819bd6-fcec-41bf-8190-b47d6c454d81 commented 2 years ago
BPO 45626
Files
  • 16155.eml: Example email
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields: ```python assignee = None closed_at = None created_at = labels = ['3.8', 'type-bug', 'library'] title = 'Email part with content type message is multipart.' updated_at = user = 'https://bugs.python.org/jdhowroyd' ``` bugs.python.org fields: ```python activity = actor = 'jdhowroyd' assignee = 'none' closed = False closed_date = None closer = None components = ['Library (Lib)'] creation = creator = 'jdhowroyd' dependencies = [] files = ['50396'] hgrepos = [] issue_num = 45626 keywords = [] message_count = 1.0 messages = ['405091'] nosy_count = 1.0 nosy_names = ['jdhowroyd'] pr_nums = [] priority = 'normal' resolution = None stage = None status = 'open' superseder = None type = 'behavior' url = 'https://bugs.python.org/issue45626' versions = ['Python 3.8'] ```

    5d819bd6-fcec-41bf-8190-b47d6c454d81 commented 2 years ago

    From the library documentation, it is an intended feature that an email part with content_maintype == "message" is treated as multipart. This does not seem to be compliant to MIME specification nor expected semantics. The attached email (from the dnc wikileaks collection) is a good example where this logic breaks down.

    Code: import pathlib pth = "16155.eml" # path to example import email import email.parser parser = email.parser.BytesParser() fl = pth.open("rb") eml = parser.parse(fl) pts = [p for p in eml.walk()] len(pts) # returns 52

    Here pts[0] is the root part of content_type 'multipart/report'. Then pts[1] has content_type 'multipart/alternative' containing the 'text/plain' pts[2] and the 'text/html' pts[3] (which most email clients would consider the message (of this email). All good so far.

    The problem is that pts[4] of content_type 'message/delivery-status' which has pts[4].is_multipart() [True] and contains 46 sub parts as returned by pts[4].get_payload(): these are pts[5], ... , pts[50]. Finally, pts[51] has content_type 'text/rfc822-headers' which is fine.

    Each of the subparts of pts[4] (pts[5:51]) have "" returned by pts[n].get_payload() as their content is treated as headers. Where as pts[4].as_bytes includes the header (and separating blank line) for that part; namely, b'Content-Type: message/delivery-status\n\n'.

    Looking at the raw file and in particular the MIME boundary makers it would seem to me that pts[4] should not be considered multipart and that there is no indication from a content-type of 'message/delivery-status' should or even could be considered an (rfc822) email.

    Moreover, as the main developer of a system to forensically analyse large (million +) corpora of emails this behaviour of treating parts even of the specific content-type 'message/rfc822' is undisarable; typically, these occur as part of bounce messages and have their content-disposition set to 'attachment'. As a developer what would seem more natural in the case that this behaviour is wanted would be to test parts for the content-type 'message/rfc822' and pass the .get_payload(decode=True) to the bytes parser parser.parse() method.

    I appreciate the need to support backwards compatibility, so presumably this would require an addition to email.policy to govern which parts should be treated as multipart. I would be more than happy to submit a patch for this but fear it would be rejected out of hand (as the original intent is clearly to parse out contained emails).