http: extend HttpEntity model to with file-based data sources/targets

[sirthias]

Currently the HttpEntity is modeled to be either an EmptyEntity or an HttpBody:

case class HttpBody(contentType: ContentType, buffer: Array[Byte])

whereby the buffer of an HttpBody is guaranteed to be non-empty.

Proposal: change the HttpBody to this:

case class HttpBody(contentType: ContentType, data: NonEmptyHttpData)

whereby the data are modeled like this:

sealed abstract class HttpData

case object EmptyHttpData extends HttpData

sealed abstract class NonEmptyHttpData extends HttpData

// bytes must be non-empty
case class HttpBytes(bytes: akka.util.ByteString) extends NonEmptyHttpData

// file must exist, be readable and non-empty
// len == 0 means: all remaining
case HttpFileBytes(file: File, offset: Long = 0, len: Long = 0) extends NonEmptyHttpData

case CompoundHttpData(head: NonEmptyHttpData, tail: HttpData) extends NonEmptyHttpData

We should have convenience constructors (HttpData.apply) for easily wrapping a ByteString or File, which provide the emptyness check and return either EmptyHttpData or an NonEmptyHttpData incarnation.

The MessageChunk can then also be changed from this:

case class MessageChunk(body: Array[Byte], extension: String)

to this

case class MessageChunk(data: NonEmptyHttpData, extension: String)

This allow us to make use of the new WriteFile command available with the new Akka IO layer and have the spray-can layer send file content without the need for loading it into the JVM heap first.

Additionally we could use it on the incoming side as well and have spray-can dump incoming data right into a file and deliver the message data as a HttpFileBytes instance (size threshold and file system location would be configurable).

[RajivKurian]

What would case CompoundHttpData(head: NonEmptyHttpData, tail: HttpData) extends NonEmptyHttpData be used for?

[sirthias]

It would allow you mix content that you application provides in-memory with content coming from files. This might be useful, for example, for producing multipart message entities.

[RajivKurian]

Would this be based on http://tools.ietf.org/html/rfc2387 ?

[sirthias]

We already have support for (un)marshalling multipart messages. Check out this thread for more background on this ticket: https://groups.google.com/forum/#!searchin/spray-user/wall/spray-user/oS0GNXHSMJU/uIFDFkq42moJ