Investigate batching for download body

[waahm7]

Investigate if batching improves performance for download body

e.g. recv_many is likely easiest thing to test

If it does we'll need to decide how to expose this especially w.r.t returning metadata per/chunk or not

[waahm7]

I have looked into this. We are already batching the 8MB parts using the collect function at https://github.com/awslabs/aws-s3-transfer-manager-rs/blob/main/aws-s3-transfer-manager/src/operation/download/service.rs#L66.

• The 8MB bytes will be SegmentedBuf, and we can potentially copy it to sequential memory, but the benchmark runner currently just looks at the size and doesn't read the SegmentedBuf for RAM workloads, so this is likely not the performance concern.
• Another thing I tried was out of order body without sorting, and here are the numbers:

  
  Out of order, without sorting:
  250 concurrency:
  Run:1 Secs:9.193960 Gb/s:28.029058
  Run:2 Secs:5.125541 Gb/s:50.277233
  Run:3 Secs:7.114564 Gb/s:36.221197
  Run:4 Secs:4.790875 Gb/s:53.789352
  Run:5 Secs:3.334504 Gb/s:77.282264
  Run:6 Secs:3.407275 Gb/s:75.631707
  Run:7 Secs:3.259767 Gb/s:79.054133
  Run:8 Secs:3.334377 Gb/s:77.285213
  Run:9 Secs:3.274595 Gb/s:78.696166
  Run:10 Secs:3.370471 Gb/s:76.457570
  125 concurrency:
  Run:1 Secs:5.179713 Gb/s:49.751415
  Run:2 Secs:6.452571 Gb/s:39.937262
  Run:3 Secs:3.490095 Gb/s:73.836961
  Run:4 Secs:3.965827 Gb/s:64.979650
  Run:5 Secs:3.784585 Gb/s:68.091483
  Run:6 Secs:3.630378 Gb/s:70.983809
  Run:7 Secs:3.859048 Gb/s:66.777623
  Run:8 Secs:3.536690 Gb/s:72.864181
  Run:9 Secs:3.627387 Gb/s:71.042332
  Run:10 Secs:3.780014 Gb/s:68.173835

main: 125 concurrency: Run:1 Secs:9.860658 Gb/s:26.133961 Run:2 Secs:3.347658 Gb/s:76.978597 Run:3 Secs:3.340746 Gb/s:77.137875 Run:4 Secs:3.595723 Gb/s:71.667929 Run:5 Secs:3.553175 Gb/s:72.526133 Run:6 Secs:3.576855 Gb/s:72.045981 Run:7 Secs:6.723821 Gb/s:38.326132 Run:8 Secs:3.584865 Gb/s:71.885011 Run:9 Secs:3.611786 Gb/s:71.349191 Run:10 Secs:3.736236 Gb/s:68.972631

250 concurrency: Run:1 Secs:10.794185 Gb/s:23.873783 Run:2 Secs:10.411145 Gb/s:24.752133 Run:3 Secs:12.076494 Gb/s:21.338812 Run:4 Secs:3.606566 Gb/s:71.452459 Run:5 Secs:12.080437 Gb/s:21.331847 Run:6 Secs:3.713954 Gb/s:69.386437 Run:7 Secs:3.604587 Gb/s:71.491700 Run:8 Secs:8.970625 Gb/s:28.726878 Run:9 Secs:3.819096 Gb/s:67.476188 Run:10 Secs:3.624001 Gb/s:71.108706

It feels a bit more stable, but we still have the same problem of slower first few runs, and overall performance is still similar. Based on this, I think delivering the body is not our current bottleneck. I haven't tested `recv_many` yet, but we can discuss if we still need to test batching multiple parts.