Parallel preparation

[TingDaoK]

• Introduce a new API that can handle reading in parallel from it.
• Added an implementation to create parallel input stream from file
• Changed the S3 client to create parallel input stream if the upload source is a file
• Use all the io eventloop group to prepare request if parallel input stream is available

Increased the upload performance if the client is handling less meta request then the number of threads we created for io event.

For a single upload of a 30GiB file, we increased the speed to read the whole read from 14 secs to around 6 secs by this PR.

Quick thinks:

• Why are we getting 6 secs to read from disk(cached)? Pure reading from cached file with 8 thread can do faster
  • Our client doesn't read the file as fast as possible, reading from file is probably NOT the bottle neck if we read in parallel and we have the file cached.
• Why use 8 threads to read?
  • From my testing, 8 threads can read the file in 236 Gbps, faster than the network bandwidth we have, so, it's a reasonable number to avoid file read as the bottle neck.
  • I tested to use 32 threads to read, I don't see any improvement from our client. The tracing shows the IO threads still worked around 6 secs. Again, I think with parallel read and cached file, the reading is no longer the bottle neck.
  • From my testing with just reading, around 32 threads will be able to reach the best performance. However, even with 1 thread, the reading throughput is higher than what we currently have.

    [ec2-user@ip-172-31-14-35 build]$ ./s3-benchrunner-c 1
    Total execution time: 4.76 seconds
    total read_Gib 240
    Read through_put: 50.38 Gbps
    [ec2-user@ip-172-31-14-35 build]$ ./s3-benchrunner-c 8
    Total execution time: 1.01 seconds
    total read_Gib 240
    Read through_put: 236.49 Gbps
    [ec2-user@ip-172-31-14-35 build]$ ./s3-benchrunner-c 16
    Total execution time: 0.56 seconds
    total read_Gib 240
    Read through_put: 424.87 Gbps
    [ec2-user@ip-172-31-14-35 build]$ ./s3-benchrunner-c 32
    Total execution time: 0.51 seconds
    total read_Gib 240
    Read through_put: 471.96 Gbps

• Will parallel read affect multiple meta requests?
  • I did a test with 24 concurrent upload with the same 30 GiB file. The throughput doesn't change much. From the main branch, I got ~54Gbps, from this branch, I got ~54Gbps as well. Note: the max-throughput benchmark, I got ~55Gbps

TODO:

• ~Instead of open file descriptor multiple times, can we use mmap instead? Will it help to improve the throughput?~ -> https://github.com/awslabs/aws-c-s3/pull/354
• ~research on mmap across platforms, and use case (huge file with limit memory). Limit the memory usage?~ -> https://github.com/awslabs/aws-c-s3/pull/354
• Unknown content-length with parallel read
• Skipping parts should still read the parts to make sure we keep the durability check.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[codecov-commenter]

Codecov Report

Merging #353 (2a0626c) into main (00a1129) will decrease coverage by 0.11%. The diff coverage is 87.85%.

Additional details and impacted files

[](https://app.codecov.io/gh/awslabs/aws-c-s3/pull/353?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=awslabs) ```diff @@ Coverage Diff @@ ## main #353 +/- ## ========================================== - Coverage 89.56% 89.45% -0.11% ========================================== Files 17 18 +1 Lines 4933 5035 +102 ========================================== + Hits 4418 4504 +86 - Misses 515 531 +16 ``` | [Files Changed](https://app.codecov.io/gh/awslabs/aws-c-s3/pull/353?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=awslabs) | Coverage Δ | | |---|---|---| | [source/s3.c](https://app.codecov.io/gh/awslabs/aws-c-s3/pull/353?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=awslabs#diff-c291cmNlL3MzLmM=) | `96.15% <ø> (ø)` | | | [source/s3\_request\_messages.c](https://app.codecov.io/gh/awslabs/aws-c-s3/pull/353?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=awslabs#diff-c291cmNlL3MzX3JlcXVlc3RfbWVzc2FnZXMuYw==) | `73.75% <ø> (-1.02%)` | :arrow_down: | | [source/s3\_parallel\_read\_stream.c](https://app.codecov.io/gh/awslabs/aws-c-s3/pull/353?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=awslabs#diff-c291cmNlL3MzX3BhcmFsbGVsX3JlYWRfc3RyZWFtLmM=) | `82.65% <82.65%> (ø)` | | | [source/s3\_auto\_ranged\_put.c](https://app.codecov.io/gh/awslabs/aws-c-s3/pull/353?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=awslabs#diff-c291cmNlL3MzX2F1dG9fcmFuZ2VkX3B1dC5j) | `92.25% <100.00%> (+<0.01%)` | :arrow_up: | | [source/s3\_default\_meta\_request.c](https://app.codecov.io/gh/awslabs/aws-c-s3/pull/353?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=awslabs#diff-c291cmNlL3MzX2RlZmF1bHRfbWV0YV9yZXF1ZXN0LmM=) | `94.59% <100.00%> (ø)` | | | [source/s3\_meta\_request.c](https://app.codecov.io/gh/awslabs/aws-c-s3/pull/353?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=awslabs#diff-c291cmNlL3MzX21ldGFfcmVxdWVzdC5j) | `92.70% <100.00%> (+0.08%)` | :arrow_up: | ... and [1 file with indirect coverage changes](https://app.codecov.io/gh/awslabs/aws-c-s3/pull/353/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=awslabs)