feat(extras): Add an S3 Multipart Upload helper

[danopia]

Comparable in outcome to the SDK's AWS.S3.ManagedUpload class.

Fixes #30

Example

import { ApiFactory } from 'https://deno.land/x/aws_api/client/mod.ts';
import { S3 } from 'https://deno.land/x/aws_api/services/s3/mod.ts';
// The new module:
import { managedUpload } from 'https://deno.land/x/aws_api/extras/s3-upload.ts';

const s3 = new ApiFactory().makeNew(S3);

const {VersionId} = await managedUpload(s3, {
    Bucket: 'my-bucket',
    Key: 'my-key',
    Body: await Deno.open('data.bin').then(x => x.readable),
});

Remaining work

• [ ] Unit tests
• [ ] Report progress? Number of parts uploaded so far?
• [x] Parallelization of part upload (default 4)
• [x] Maybe refactor to use streams better?

[TillaTheHun0]

@danopia I was able to pull that helper in and it works, nicely done! I am going to run it through some load to see if there are any noticeable performance improvements.

[danopia]

Excellent, thanks for the validation effort ❤️

In my quick tests I noticed that memory usage isn't excellent. It seems that aws-api makes a copy or two of each request payload, so the default 20mb buffer (5mb*4) can actually become 60mb or so. That's not related to this helper, so I'll probably look at optimizing memory in a different PR.

[TillaTheHun0]

@danopia performance did seem to improve, in particular for larger files approaching the 1-2 MB size. Though that would make sense. Pretty sweet!

Memory usage did seem to increase, but CPU increased more in the tests that I ran.

This was running on a container on Fargate with 1 vCPU and 2048 GB of memory, fwiw.

[danopia]

I'll merge this in the coming days (just a couple more unit tests) and then also verify with some of the other S3 providers before tagging a release.

Note that the function is now named managedUpload to match AWS SDK, because it only multi-parts on files larger than 5MB so the name was misleading in the end.