search

awslabs / aws-c-s3

C99 library implementation for communicating with the S3 service, designed for maximizing throughput on high bandwidth EC2 instances.
Apache License 2.0
93 stars 37 forks source link

Allow disabling upload checksums while using upload review #421

Closed jamesbornholt closed 4 months ago

jamesbornholt commented 5 months ago

Some clients (Mountpoint) might want to disable actually sending upload checksums when talking to S3 endpoints that don't support the additional checksums feature. But they still want to use the upload review callback to at least check that the parts sent matched the checksums they expect, to detect corruption while parts were sitting in CRT memory before being sent. This change makes it possible to set a checksum location of AWS_SCL_NONE while still computing checksums for each upload part. The upload review callback will be invoked as normal, including per-part checksums, but the checksums won't be included in the requests sent to S3.

(My C is pretty rusty these days, so, uh, feedback welcome)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

codecov-commenter commented 4 months ago

Codecov Report

Attention: Patch coverage is 80.55556% with 7 lines in your changes are missing coverage. Please review.

Project coverage is 89.52%. Comparing base (3334843) to head (4efc4e4).

Additional details and impacted files [![Impacted file tree graph](https://app.codecov.io/gh/awslabs/aws-c-s3/pull/421/graphs/tree.svg?width=650&height=150&src=pr&token=J4KP54FVLF&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=awslabs)](https://app.codecov.io/gh/awslabs/aws-c-s3/pull/421?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=awslabs) ```diff @@ Coverage Diff @@ ## main #421 +/- ## ========================================== - Coverage 89.63% 89.52% -0.12% ========================================== Files 20 20 Lines 5992 6012 +20 ========================================== + Hits 5371 5382 +11 - Misses 621 630 +9 ``` | [Files](https://app.codecov.io/gh/awslabs/aws-c-s3/pull/421?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=awslabs) | Coverage Δ | | |---|---|---| | [source/s3\_auto\_ranged\_put.c](https://app.codecov.io/gh/awslabs/aws-c-s3/pull/421?src=pr&el=tree&filepath=source%2Fs3_auto_ranged_put.c&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=awslabs#diff-c291cmNlL3MzX2F1dG9fcmFuZ2VkX3B1dC5j) | `92.77% <100.00%> (+0.02%)` | :arrow_up: | | [source/s3\_client.c](https://app.codecov.io/gh/awslabs/aws-c-s3/pull/421?src=pr&el=tree&filepath=source%2Fs3_client.c&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=awslabs#diff-c291cmNlL3MzX2NsaWVudC5j) | `88.15% <100.00%> (ø)` | | | [source/s3\_request\_messages.c](https://app.codecov.io/gh/awslabs/aws-c-s3/pull/421?src=pr&el=tree&filepath=source%2Fs3_request_messages.c&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=awslabs#diff-c291cmNlL3MzX3JlcXVlc3RfbWVzc2FnZXMuYw==) | `73.97% <84.21%> (+0.22%)` | :arrow_up: | | [source/s3\_checksum\_stream.c](https://app.codecov.io/gh/awslabs/aws-c-s3/pull/421?src=pr&el=tree&filepath=source%2Fs3_checksum_stream.c&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=awslabs#diff-c291cmNlL3MzX2NoZWNrc3VtX3N0cmVhbS5j) | `74.07% <71.42%> (-0.40%)` | :arrow_down: | ... and [2 files with indirect coverage changes](https://app.codecov.io/gh/awslabs/aws-c-s3/pull/421/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=awslabs)