email header indentation destroyed

[2f6fc7b2-6580-4d69-b323-71413f9f4219]

BPO	27256
Nosy	@warsaw, @bitdancer
Files	utf8-header-failure.mail: mail as read from email.message_from_binary_file mf.9__mi0bf.out: resulting mail from BytesGenerator.flatten email_flatten.py

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields: ```python assignee = None closed_at = None created_at = labels = ['type-bug', 'expert-email'] title = 'email header indentation destroyed' updated_at = user = 'https://bugs.python.org/frispete' ``` bugs.python.org fields: ```python activity = actor = 'frispete' assignee = 'none' closed = False closed_date = None closer = None components = ['email'] creation = creator = 'frispete' dependencies = [] files = ['43285', '43286', '43417'] hgrepos = [] issue_num = 27256 keywords = [] message_count = 3.0 messages = ['267732', '267916', '268675'] nosy_count = 3.0 nosy_names = ['barry', 'frispete', 'r.david.murray'] pr_nums = [] priority = 'normal' resolution = None stage = 'test needed' status = 'open' superseder = None type = 'behavior' url = 'https://bugs.python.org/issue27256' versions = ['Python 3.5', 'Python 3.6'] ```

[2f6fc7b2-6580-4d69-b323-71413f9f4219]

In the course of replacing an old Python 2.7 email filter tool with a rewritten Python3 version, I stumbled across a ugly case, where such an header:

X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtCTDJQUjAyTUI1MTQ7MjM6bEtRRlNaUHQvVTk5WCttdktlOUVrUGQvVFBH?= =?utf-8?B?cDFJemVUeXFzOGNzYnZOYWlwMDZpR0YzbXZyY09WaTBKM2pkeUl4S1VDMkxw?= =?utf-8?B?eVRkNWthRW9waUhJTzczTWd5WDZOQ3hMNU1haGFvQTVzVTdRZmxJUnZlblpW?= ...

is regenerated as:

X-Microsoft-Exchange-Diagnostics: 1;BL2PR02MB514;23:lKQFSZPt/U99X+mvKe9EkPd/TPG p1IzeTyqs8csbvNaip06iGF3mvrcOVi0J3jdyIxKUC2Lp yTd5kaEopiHIO73MgyX6NCxL5MahaoA5sU7QflIRvenZV

which is plain wrong of course.

I'm using email.message_from_binary_file for parsing and BytesGenerator.flatten for regeneration. Since those are LKML public mails, I'm attaching both versions.

I'm using 3.4.4, but also the email module from current hg for testing.

[bitdancer]

Hmm. There appear to be at least two bugs here, using the SMTP policy. I thought I had test cases like this, but clearly I don't :(

[2f6fc7b2-6580-4d69-b323-71413f9f4219]

Sorry guys for not providing this earlier.

It turned out, that the sub optimal behaviour is related to a unfortunate policy choice: email.policy.SMTP.