Support for manual resumption of multipart uploads

[konklone]

One use case I'd love to make happen is allowing files to be uploaded over multiple sessions. If I can store the UploadId of a multipart upload (and the offset/part that was last uploaded), and pause the upload without either completing or aborting it, I should be able to come back any time later and resume the upload.

I think this library doesn't offer any facilities for that right now. I can take a look about adding support myself the next time I'm working on this stuff, but any thoughts or advice or caveats welcome.

[nathanpeck]

I think the best way to do this would be to add optional parameters to the upload stream constructor to allow you to manually specify an upload ID and a list of upload part ID's that have already been uploaded, then use fs.createReadStream() with the start and end options to create a readstream that starts from the correct position where the uploaded stopped, and goes to the end of the file.

I'm not sure how well/if fs.createReadStream() is supported in browserify, however. Another less sophisticated and more wasteful solution would be to just read in the entire source file from the beginning but throw away the data until it reaches the part that has not been uploaded yet.

However, that is obviously a less ideal solution compared to a proper implementation that starts reading the file at the correct location.

[konklone]

I think the best way to do this would be to add optional parameters to the upload stream constructor to allow you to manually specify an upload ID and a list of upload part ID's that have already been uploaded,

Yeah, okay, that sounds right. I'll work on that.

then use fs.createReadStream() with the start and end options to create a readstream that starts from the correct position where the uploaded stopped, and goes to the end of the file.

I'm using filereader-stream for bridging to the FileReader API, and it supports starting at a particular byte offset. So I should be fine on that front.

[1N50MN14]

+1

When I made the PR to filereader-stream to add support for pause/resume I had this use case on my mind. filereader-stream also emits a progress event by the way which is useful in this case. Looking forward to seeing what you come up with @konklone.

[konklone]

I'm working on this now, branched off of 1.0.5, since that hasn't been merged into master yet (though it is out on npm). I don't want to submit a PR back to a tagged branch, so I'll plan to submit it to master.

[konklone]

This work is now in https://github.com/nathanpeck/s3-upload-stream/pull/25. Closing in favor of resolving things there.