Add support for more flexible backend streams

[ppetr]

I wanted to use IMAP with TLS and this turned out to be quite complex. (Relates to #7.) So far, I found these solutions:

• Use threads - one for wrapping a connection into TLS and another for the actual IMAP handling. This is awful.

• Use MonadIO instead of IO everywhere. I already tried this variant in this branch for IMAP. It seems to work well and I was able to fuse such patched IMAP with network-conduit. The main part looks like this:

  main = runTCPClient (clientSettings 143 (BS.pack "imap.centrum.cz"))
            capabilities
  
  capabilities :: Application IO
  capabilities ad = appSource ad $= f $$ appSink ad
  where
    f :: Conduit BS.ByteString IO ByteString
    f = do
            c <- connectStream conduitBSStream
            capability c >>= liftIO . print
            logout c
  
  type BSPipe m = ConduitM ByteString ByteString m
  
  conduitBSStream :: (Monad m) => BSStreamM (BSPipe m)
  -- implementation ...

  Here BSStreamM is a (backward compatible) modification that takes an arbitrary MonadIO instead of IO. This allows us to run the whole code in Conduit instead of IO and so the actions in BSStreamM can be operations constructed using Conduit primitives. (Full example code here).

  This modification adds generality and the only loss is slightly more complex type signatures.

• Modify HaskellNet so that it's based on conduit instead of IO. I don't have a clear picture yet. The general idea is that connection wouldn't hold a BSStream but a Sink for network output and a ResumableSource for network input. This would allow to plug in any conduit, even a pure one (for testing, for example).

I'm willing to participate on those changes, if we reach some consensus.

[mejrpete]

This is something that I would love to see implemented. It would do wonders for a project that I was participating on that was more or less abandoned a while ago because of difficulties with this and related issues.

I'd also be willing to participate if I can be of any help.

[jtdaugherty]

Great! I don't have a strong preference as to which approach is taken. I also don't have time to work on this myself right now, so I'm going to be especially appreciative of anyone who wants to take this on. :) With that said, the MonadIO approach seems like a good middle ground for now until something clearer with e.g. conduit arises. As far as I can tell, there is not a community consensus on which iterative I/O approach to use, anyway. Regardless: if you want to hack up a conduit implementation just to see how it might go, that would be great. Or you can just submit a pull request for the MonadIO approach and have done with it.

[ppetr]

@jtdaugherty It seems that converting to MonadIO is just enough so that it's possible to support conduit. I hacked a function that converts a conduit into a MonadIO variant of BSStream:

conduitToStream
    :: (MonadIO m)
    => Source m ByteString
    -> Sink (Flush ByteString) m ()
    -> m (BSStreamM m)

Similarly, I constructed

tlsContextConduit
    :: (MonadIO m)
    => Context
    -> (Source m ByteString, Sink (Flush ByteString) m ())

for TLS contexts from Network.TLS. Combined together, I got IMAP working on TLS.

One problem is that it doesn't really make sense to close a conduit. Another conduit can be appended after it and continue its processing. This can be also viewed as that closing a connection is the responsibility of whoever created it, not IMAP's. So currently for conduits bsClose does nothing and bsIsOpen return always True. Any ideas?

With this structure, I think we could

1. Patch HaskellNet to use MonadIO everywhere.
2. Add a new package HaskellNet-conduit that will depend on conduit and provide BSStreams from conduits.

[jtdaugherty]

I think having a separate integration package like HaskellNet-conduit makes a lot of sense. As for the specifics of how best to go about doing it, I really couldn't say. :)