MIGRATION ISSUE: Equivalent way of disabling MD5 (S3DisableContentMD5Validation)

stanhu commented 3 months ago

Pre-Migration Checklist

[X] I've read the Migration Guide.
[X] I've checked AWS Forums and StackOverflow for similar migration issues.

Go Version Used

Go 1.21

Describe the Migration Issue

In AWS SDK Go v1, we previously used S3DisableContentMD5Validation to avoid computing MD5 checksums in the PutObject API call in FIPS systems:

From https://docs.aws.amazon.com/sdk-for-go/api/aws/:

    // S3DisableContentMD5Validation config option is temporarily disabled,
    // For S3 GetObject API calls, #1837.
    //
    // Set this to `true` to disable the S3 service client from automatically
    // adding the ContentMD5 to S3 Object Put and Upload API calls. This option
    // will also disable the SDK from performing object ContentMD5 validation
    // on GetObject API calls.
    S3DisableContentMD5Validation *bool

I saw the comment in https://github.com/aws/aws-sdk-go-v2/issues/1040#issuecomment-1051054781, but I'm not sure if this meant it was possible to disable MD5 outright.

Could someone clarify what needs to be done here?

Code Comparison

V1:

    cfg := &aws.Config{
        S3DisableContentMD5Validation: aws.Bool(True),
    }

Observed Differences/Errors

There's no obvious way to set S3DisableContentMD5Validation.

Additional Context

No response

RanVaknin commented 2 months ago

Hi @stanhu ,

Is there a particular operation that calculates md5 for you? In v2 md5 shouldn't be calculated by default.

v1:

PUT /hey123.txt HTTP/1.1
Host: testbucket-REDACTED.s3.amazonaws.com
User-Agent: aws-sdk-go/1.50.9 (go1.19.1; darwin; arm64)
Content-Length: 3
Authorization: AWS4-HMAC-SHA256 Credential=REDACTED/20240904/us-east-1/s3/aws4_request, SignedHeaders=content-length;content-md5;host;x-amz-content-sha256;x-amz-date, Signature=REDACTED
Content-Md5: rL0Y20zC+Fzt72VPzMSk2A==
X-Amz-Content-Sha256: REDACTED
X-Amz-Date: 20240904T165439Z
Accept-Encoding: gzip

v2:

PUT /hey123.txt?x-id=PutObject HTTP/1.1
Host: testbucket-REDACTED.s3.us-west-2.amazonaws.com
User-Agent: m/E aws-sdk-go-v2/1.30.4 os/macos lang/go#1.19.1 md/GOOS#darwin md/GOARCH#arm64 api/s3#1.61.0
Content-Length: 13
Accept-Encoding: identity
Amz-Sdk-Invocation-Id: REDACTED
Amz-Sdk-Request: attempt=1; max=3
Authorization: AWS4-HMAC-SHA256 Credential=REDACTED/20240904/us-west-2/s3/aws4_request, SignedHeaders=accept-encoding;amz-sdk-invocation-id;amz-sdk-request;content-length;content-type;host;x-amz-content-sha256;x-amz-date, Signature=REDACTED
Content-Type: application/octet-stream
X-Amz-Content-Sha256: UNSIGNED-PAYLOAD
X-Amz-Date: 20240904T165749Z

You can view this by enabling the network logger:

    cfg, err := config.LoadDefaultConfig(context.TODO(), config.WithRegion("us-west-2"), config.WithClientLogMode(aws.LogRequestWithBody|aws.LogResponseWithBody))

Thanks, Ran~

stanhu commented 2 months ago

Ah, even better. Thanks!

github-actions[bot] commented 2 months ago

This issue is now closed. Comments on closed issues are hard for our team to see. If you need more assistance, please open a new issue that references this one.

aws / aws-sdk-go-v2