[benchmark] The 3 methods in cfs-s3

[raix]

Testing 3 different writeStream / upload patterns for s3 So this is the ground work for picking the best way of creating a write stream for the s3 SA

1. the s3.indirect.streaming.js loads the write stream into temp storage to get the content length then uses the putObject streaming api for uploading the image
2. s3.upload.stream.js this loads the data from the write stream into a buffer and slices data chunks out and sends it the the s3 server
3. s3.upload.stream2.js this one is streaming the writes and buffering up data to min. 5mb chunks then uploads the chunk. This only buffers a max of one chunk (min 5mb)

I’m not sure which is best - 1 or 3 because 2 buffers the whole file in memory and got a lot of dependencies.

• 1. because its using the filesystem stream and connects it directly to the s3.putObject stream api - and everything is handled for us
• 3. is perhaps the best because we skip the filesystem temp file - otherwise basically the same stuff.

It could be fun to benchmark and test how much memory is actually used for these 3 methods.

Reference

[aldeed]

We should probably stick with option 3 (currently used) and delete the others.