aksiksi / vaulty

A service that allows users to send email directly to a cloud storage service.
https://vaulty.net
4 stars 0 forks source link

Stream attachments directly to storage #8

Closed aksiksi closed 4 years ago

aksiksi commented 4 years ago

Reqwest implements From<Response> for Body. This means that a request body can be set for a pending response. It is not clear if the content is streamed or not.

For Attachment, we can set the content filed to point to an enum like so:

enum AttachmentBody {
    Data(Vec<u8>),
    Stream(reqwest::Response),
}

Note that you can convert a Response into a Stream, and then pass that on. Or we can use the chunk method. See: https://docs.rs/reqwest/0.10.0/reqwest/struct.Response.html#method.bytes_stream

aksiksi commented 4 years ago

For each attachment:

  1. Pass in sender, recipients, content type, email ID, and filename as HTTP headers.
  2. Set attachment body as the body.
  3. On server side, validate headers and return relevant error (graceful or otherwise).
  4. Stream HTTP body to Dropbox with given filename.
aksiksi commented 4 years ago

I did some basic profiling (release build) to compare approaches. For an ~8 MB email with 3 attachments:

See graphs (courtesy of massif-visualizer):

old new

See https://github.com/seanmonstar/warp/issues/448 for how to use this.