search

eclipse-edc / Technology-Aws

Apache License 2.0
5 stars 15 forks source link

fix(s3 dataplane): Fix transfer of empty objects #417

Closed rafaelmag110 closed 3 weeks ago

rafaelmag110 commented 3 weeks ago

What this PR changes/adds

When creating a folder via the AWS console, a 0 byte object with the name of the folder is created. This object is picked up by the s3 listObjects call and passed to transferParts as a valid part to be transfered. This creates an unnecessary Part to be uploaded which can be filtered out.

Also, the multipart upload logic wasn't able to upload 0 byte files since no empty completedParts list can be used in a completedMultipartUpload request.

Why it does that

To enable the transfer of 0 byte files and remove the unecessary upload of folder 0 byte marker.

Further notes

Some tests where adapted to best represent the testing cases.

Linked Issue(s)

Closes #384

codecov-commenter commented 3 weeks ago

:warning: Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

Attention: Patch coverage is 66.66667% with 2 lines in your changes missing coverage. Please review.

Project coverage is 65.53%. Comparing base (d177a98) to head (900004a). Report is 63 commits behind head on main.

Files Patch % Lines
...pse/edc/connector/dataplane/aws/s3/S3DataSink.java 75.00% 0 Missing and 1 partial :warning:
...e/edc/connector/dataplane/aws/s3/S3DataSource.java 50.00% 0 Missing and 1 partial :warning:

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #417 +/- ## ============================================ + Coverage 63.82% 65.53% +1.70% - Complexity 0 117 +117 ============================================ Files 26 28 +2 Lines 633 676 +43 Branches 30 32 +2 ============================================ + Hits 404 443 +39 + Misses 222 218 -4 - Partials 7 15 +8 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

rafaelmag110 commented 3 weeks ago

While discussing the changes with @bmg13 we identified another case where having nested folder files would brake the tests. We accounted for that case in the last commit.

@paullatzelsperger This is ready for review. I can't add properly request the review...