Multipart base64 encoding produces long line

[manuco]

Long story short

When using the aiohttp.multipart module, and its MultipartWriter class, we can create multipart documents whose payload can be transfert-encoded in base64.

We use this module to retrieve some http content via aiohttp client, and stream a multipart document containing the Responses provided by the client to another processus via stdout (so we're not buffering the contents into memory, but act like a proxy).

The MultipartWriter generates base64 subdocuments without newlines. In version 0.2.1 of aiohttp, it was with multiple lines.

I know the base64 format does not define line length, but :

• there are very few reasons to use the multipart module with base64 encoding in http transaction (since http can handle binary contents)
• there are very good reasons to use multipart + base 64 in emails since emails are 7 bits only
• emails format is described in RFC5322
• RFC5322 states that line length must be more than 998 characters (and should not be more than 78 characters) (https://tools.ietf.org/html/rfc5322.html#section-2.1.1)
• RFC2045, which define MIME (englobing the multipart format), states that "The encoded output stream must be represented in lines of no more than 76 characters each. " (https://tools.ietf.org/html/rfc2045.html#section-6.8)

I think the MultipartWriter should allow us to generate base64 sub contents with newlines (potentially with an option).

Expected behaviour

When producing a multipart document with base64 tranfert encoding, the final document should not have lines longer than 78 characters (more or less).

Actual behaviour

When producing a multipart document with base64 tranfert encoding, the final document has the entire base64 subdocument on one line.

Steps to reproduce

import sys

import asyncio
import aiohttp.multipart as mp

class Writer:
    @asyncio.coroutine
    def write(self, chunk):
        sys.stdout.buffer.write(chunk)

a = mp.MultipartWriter()
a.append("i" * 200, {"Content-transfer-encoding": "base64"})
loop = asyncio.get_event_loop()

loop.run_until_complete(a.write(Writer()))

Fix

In aiohttp.multipart.MultipartPayloadWriter.write() method, the call to base64.b64encode could be replaced by base64.encodebytes which takes care of the newlines.

[fafhrd91]

I think this is reasonable. would you create PR and test case?

[manuco]

I'll try this. No deadline for now, but I'm thinking of it.