search

aws / aws-sdk-cpp

AWS SDK for C++
Apache License 2.0
1.98k stars 1.06k forks source link

S3Crt fails when downloading exactly 5242881 bytes from s3 bucket #2704

Closed aschuess closed 1 year ago

aschuess commented 1 year ago

Describe the bug

When using the Aws::S3Crt::Model::GetObjectRequest and GetObject API

on an object that is precisely ((5 1024 1024) + 1 bytes - only (5 1024 1024) bytes are returned.

When I download the same object through the AWS Web Console I get the full file.

Even when specifying a range of the last, say, 100 bytes, I am missing the final byte.

There was an identical bug report in the ruby sdk: https://github.com/aws/aws-sdk-ruby/issues/2561

Expected Behavior

I expect to receive all 5242881 bytes of the s3 object

Current Behavior

Only 5242880 bytes of the s3 object is returned

Reproduction Steps

  1. create a file that is 5MB + 1 byte (5242881 bytes)

  2. Upload to s3

  3. Download the object using Aws::S3Crt::S3CrtClient tooling

Possible Solution

Off-by-one bug likely around the chunk size threshold is probably the issue

Additional Information/Context

No response

AWS CPP SDK version used

1.9.359

Compiler and Version used

Gcc 9

Operating System and version

EL8

aschuess commented 1 year ago

I confirmed that changing the partSize of the ClientConfiguration makes the download of the 5M file work. (Of course files that are at the now different partSize value will no longer work)

jmklix commented 1 year ago

Working on reproducing this. ~Since this also appears in the ruby sdk it might be a problem with the service rather than the sdk.~ Either way I'm looking into getting this fixed.

jmklix commented 1 year ago

Can you try updating to the latest version of this sdk. There was an update in aws-c-s3 to fix this exact issue. Please let me know if you run into any problems or still run into this off by one error.

github-actions[bot] commented 1 year ago

Greetings! It looks like this issue hasn’t been active in longer than a week. We encourage you to check if this is still an issue in the latest release. Because it has been longer than a week since the last update on this, and in the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please feel free to provide a comment or add an upvote to prevent automatic closure, or if the issue is already closed, please feel free to open a new one.