Closed andydunstall closed 11 months ago
(I tested on the AWS SDK with no patches and that seems to fail too...)
weird. please see aws s3api put-object --help
using awscli. I remember I used its verbosity/debugging printings to dive deep into the API.
https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html
Ah oops I misunderstood how checksums work - you have to include the CRC32 in the 'create upload' request - then each CRC32 part in the final completion request (so yeah must have been only testing against MinIO and forgot to try against S3 itself...)
(Still seeing ~25% CPU improvement over MD5)
All modified lines are covered by tests :white_check_mark:
Comparison is base (
51d32b0
) 77.46% compared to head (16af4d0
) 77.46%.
:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
Using the CRC32 header works on MinIO, though running again on S3 before merging the dragonfly changes found S3 returning:
I don't understand why given it has headers:
So I don't know why it expects no checksum... I'm fairly certain it worked before, though maybe I was only testing MinIO (another argument for regression tests using S3 directly)
Will remove for now just to unblock merging the Dragonfly side (it was to improve resource usage rather than a functional requirement) - will look at next