Feature/optics and serialisation

[romanofski]

This is based on @frasertweedale's work including the serialisation of MIMEMessages

[romanofski]

@frasertweedale two remaining questions:

• I've created lenses for From, To, Date etc. Yet, I can't set the Date header for example, if the structure is missing the key. Does that mean I need a Traversal instead of a Lens? These lenses use the Mailbox and Address instances which would enforce consistency with parsing. I'd be kind of interested to use them.
• Two tests are currently failing which test the rendering. I suppose cabal uses a different current working directory and the tests have trouble to find the file comparing to. If you have a cabal setup, would you mind having a look?

Before we merge this, we'll need an additional backlog item to implement transfer encoding for 8bit (quoted printable I guess)

[frasertweedale]

@romanofski re transfer encoding, here are my ideas:

First cut: scan the whole input, if we hit anything that requires transfer encoding, short circuit and use base64. Rationale: large attachments are likely to be binary and short circuit. Text messages are likely to be small so even though we may have to scan the whole thing to learn that it does not require transfer encoding, the cost is likely small.

Alternatively: two functions, one to add ByteString to the message and unconditionally use base64 transfer encoding, the other to add Text (or a ByteString supposed to be a text message) to the message and unconditionally use QP.

Down the track, if we want to be "clever", we can:

• scan the input byte by byte

• track the ratio of "does not require transfer encoding" versus "does require transfer encoding"

• if we encounter anything that will require transfer encoding, limit the scan to 1k bytes (suggestion) or the current offset (whichever is greater)

  • if the ratio exceeds a certain threshold, base64 is indicated, otherwise use quoted-printable

Rationale:

• we don't want to scan the whole of big binary files. But we do want to scan ahead a bit to work out whether QP or Base64 seems optimal, both in terms of size, and for human readability (e.g. utf-8 text that is mostly ASCII chars should use QP so that it is "mostly readable").

[frasertweedale]

I'll look at the cabal stuff soon!

[romanofski]

Hm.. I think my misunderstanding is rooted in the transfer encoding of the message header vs. entities. Default encoding is 7bit. If field values (e.g. from address) use a non-ascii character, I can switch the message tranfer encoding to 8bit, and simply encode the field value with QP. The message body can remain unencoded. The createAttachment function already sets a tranfer-encoding (base64) for the entity. I guess that doesn't leave anything unclear. Let me see if I can already incorporate in this patch ...

[frasertweedale]

@romanofski re failing tests, the travis script current creates source dist, then extracts it and builds & runs tests there. The test-vectors/ were not in sdist hence missing file. I've pushed a fix, waiting for CI now.

edit: and we're green.

[frasertweedale]

I've created lenses for From, To, Date etc. Yet, I can't set the Date header for example, if the structure is missing the key. Does that mean I need a Traversal instead of a Lens? These lenses use the Mailbox and Address instances which would enforce consistency with parsing. I'd be kind of interested to use them.

Yes, you need a Traversal, or alternatively a Lens Headers (Maybe Date) because of the possible absence of the Date header. But a Traversal will not allow you to set the header if it is missing in the map. So Lens Headers (Maybe Date) (i.e. use the At instance) looks like the way to go.

For the other From, To and Date header the absense of the header can be "papered over" by exposing an empty list.

[frasertweedale]

I'll re-review this whole commit tomorrow. I think we can merge it pretty soon hey. Then I'll start work on the encoded-word stuff. Would be good if we can find some new contributors to fix some of the new issues we filed, some of which (e.g. time, process execution) are fairly small and isolated.

[romanofski]

Yes they are. Cheers for reviewing. Looking over the haddock examples, do you think we can enable doctest and have the examples run as tests? I fear otherwise they might go stale... (If so happy to file a backlog item)

[frasertweedale]

@romanofski merged it, push a couple of cleanup commits and a bugfix, and I'm now working on encoded-word support for address headers.

[romanofski]

Sweet thank you! I'll update the purebred side and push again.