Use mmap to for parallel read streams

[TingDaoK]

• A quick internal only impl of mmap wrapper.

Why mmap?

• We can avoid the possible FD limits with parallel read.
• We are not loading the file from disk to memory until needed
• It's "supposed" to be faster

Is mmap really better?

• ~I try to simply memcpy the mapped memory to another buffer, the OS seems to just skip loading the real data, I didn't find a way to easily ensure we load the data.~ It was a bug in my test program, now, I saw it's just the same as multi-thread fread for both cached or cold files.
• From the S3 tests, I am seeing slightly performance improvement with mmap and just fread.

  
  mmap 
  Run:1 Secs:12.621 Gb/s:20.4 Mb/s:20418.3 GiB/s:2.4 MiB/s:2434.1
  Run:2 Secs:8.083 Gb/s:31.9 Mb/s:31880.9 GiB/s:3.7 MiB/s:3800.5
  Run:3 Secs:8.686 Gb/s:29.7 Mb/s:29669.5 GiB/s:3.5 MiB/s:3536.9
  Run:4 Secs:9.205 Gb/s:28.0 Mb/s:27995.9 GiB/s:3.3 MiB/s:3337.4
  Run:5 Secs:8.259 Gb/s:31.2 Mb/s:31201.7 GiB/s:3.6 MiB/s:3719.5
  Run:6 Secs:9.032 Gb/s:28.5 Mb/s:28532.3 GiB/s:3.3 MiB/s:3401.3
  Run:7 Secs:9.140 Gb/s:28.2 Mb/s:28195.1 GiB/s:3.3 MiB/s:3361.1
  Run:8 Secs:7.305 Gb/s:35.3 Mb/s:35278.1 GiB/s:4.1 MiB/s:4205.5
  Run:9 Secs:9.986 Gb/s:25.8 Mb/s:25805.1 GiB/s:3.0 MiB/s:3076.2
  Run:10 Secs:9.243 Gb/s:27.9 Mb/s:27881.3 GiB/s:3.2 MiB/s:3323.7

fread Run:1 Secs:12.924 Gb/s:19.9 Mb/s:19940.2 GiB/s:2.3 MiB/s:2377.1 Run:2 Secs:9.986 Gb/s:25.8 Mb/s:25806.7 GiB/s:3.0 MiB/s:3076.4 Run:3 Secs:9.382 Gb/s:27.5 Mb/s:27466.3 GiB/s:3.2 MiB/s:3274.2 Run:4 Secs:10.024 Gb/s:25.7 Mb/s:25707.0 GiB/s:3.0 MiB/s:3064.5 Run:5 Secs:8.134 Gb/s:31.7 Mb/s:31683.2 GiB/s:3.7 MiB/s:3776.9 Run:6 Secs:9.643 Gb/s:26.7 Mb/s:26722.7 GiB/s:3.1 MiB/s:3185.6 Run:7 Secs:9.860 Gb/s:26.1 Mb/s:26136.3 GiB/s:3.0 MiB/s:3115.7 Run:8 Secs:9.008 Gb/s:28.6 Mb/s:28606.2 GiB/s:3.3 MiB/s:3410.1 Run:9 Secs:9.545 Gb/s:27.0 Mb/s:26998.6 GiB/s:3.1 MiB/s:3218.5 Run:10 Secs:8.926 Gb/s:28.9 Mb/s:28869.0 GiB/s:3.4 MiB/s:3441.5

- From tracing of the s3 tests, I saw the improvement from 7.5 secs to 6.8 secs, where our IO threads are doing intensive works. But, I would assume it's not really related to the difference between mmap and fread, as they are very likely not the bottle neck of the process.

How will mmap affect our memory usage?
- Needs to test and tracing. `mmap` supports to load the data on needed and we control how many data to read into memory. It "SHOULD" have no affect on our memory usage. However, I am not sure, I'll run some tracing and see the memory usage to make sure.
- Used `top` to track the memory usage, found that OS will load the data into memory and keep it until we unmap the memory. Fixed the memory issue by only map the needed part. The latest change has no effects on memory usage now.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[codecov-commenter]

Codecov Report

Merging #354 (b1b0a98) into parallel-preparation (7630d1c) will increase coverage by 0.00%. The diff coverage is 89.09%.

Additional details and impacted files

[](https://app.codecov.io/gh/awslabs/aws-c-s3/pull/354?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=awslabs) ```diff @@ Coverage Diff @@ ## parallel-preparation #354 +/- ## ===================================================== Coverage 89.45% 89.45% ===================================================== Files 18 19 +1 Lines 5035 5073 +38 ===================================================== + Hits 4504 4538 +34 - Misses 531 535 +4 ``` | [Files](https://app.codecov.io/gh/awslabs/aws-c-s3/pull/354?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=awslabs) | Coverage Δ | | |---|---|---| | [source/s3\_meta\_request.c](https://app.codecov.io/gh/awslabs/aws-c-s3/pull/354?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=awslabs#diff-c291cmNlL3MzX21ldGFfcmVxdWVzdC5j) | `92.72% <100.00%> (+0.01%)` | :arrow_up: | | [source/s3\_parallel\_read\_stream.c](https://app.codecov.io/gh/awslabs/aws-c-s3/pull/354?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=awslabs#diff-c291cmNlL3MzX3BhcmFsbGVsX3JlYWRfc3RyZWFtLmM=) | `83.15% <92.85%> (+0.50%)` | :arrow_up: | | [source/aws\_mmap.c](https://app.codecov.io/gh/awslabs/aws-c-s3/pull/354?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=awslabs#diff-c291cmNlL2F3c19tbWFwLmM=) | `87.17% <87.17%> (ø)` | |