Content-MD5 header required when using `createBucket()` method

[me-ydv-5]

Describe the bug

when trying to use s3Client.createBucket(bucketName), I get

com.amazonaws.services.s3.model.AmazonS3Exception: Content-MD5 OR x-amz-checksum- HTTP header is required for Put Object requests with Object Lock parameters

I'm not sure if I can add that header while sending the request (one way to do that is to add it via ClientConfiguration object), if yes, how do I calculate MD5 for a bucket that I want to create, there's no body as such, just a bucket name.

Expected Behavior

The request should succeed without throwing the error.

Current Behavior

Throwing the attached error

com.amazonaws.services.s3.model.AmazonS3Exception: Content-MD5 OR x-amz-checksum- HTTP header is required for Put Object requests with Object Lock parameters

Reproduction Steps

something like

var s3Client = AmazonS3ClientBuilder
        .standard()
        .withEndpointConfiguration(
            new AwsClientBuilder.EndpointConfiguration("us-east-1-endpoint", "us-east-1"))
        .withPathStyleAccessEnabled(true)
        .withClientConfiguration(new ClientConfiguration())
        .withCredentials(new AWSStaticCredentialsProvider(credentials)).build();

s3Client.createBucket("mybucket");

Possible Solution

No response

Additional Information/Context

No response

AWS Java SDK version used

1.12.276

JDK version used

openjdk version "17.0.1" 2021-10-19

Operating System and version

MacOS 13.3.1 (a)

[debora-ito]

The error message makes sense for PutObject, but not for CreateBucket.

Can you provide a functioning repro code and the full stacktrace of the error?

[me-ydv-5]

Hi, Thank you for following up on this super soon (will def help unblock me faster :)), the code would look sth like this

package com.example.test;

import com.amazonaws.ClientConfiguration;
import com.amazonaws.auth.WebIdentityTokenCredentialsProvider;
import com.amazonaws.client.builder.AwsClientBuilder;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;

public class CreateBucket {

  public static void main(String[] args) {
    WebIdentityTokenCredentialsProvider credentialsProvider = WebIdentityTokenCredentialsProvider.builder()
        .webIdentityTokenFile("path-to-token-file")
        .build();
    var s3Client = AmazonS3ClientBuilder
        .standard()
        .withEndpointConfiguration(
            new AwsClientBuilder.EndpointConfiguration("us-east-1-endpoint", "us-east-1"))
        .withPathStyleAccessEnabled(true)
        .withClientConfiguration(new ClientConfiguration())
        .withCredentials(credentialsProvider).build();

    s3Client.createBucket("mybucket");
  }
}

The stack trace is like so:

{"timestamp":{"seconds":1690898554,"nanos":143000000},"severity":"WARNING","message":"The following warnings have been detected: WARNING: Unknown HK2 failure detected:\nMultiException stack 1 of 2\ncom.amazonaws.services.s3.model.AmazonS3Exception: Content-MD5 OR x-amz-checksum- HTTP header is required for Put Object requests with Object Lock parameters (Service: Amazon S3; Status Code: 400; Error Code: InvalidRequest; Request ID: <reqID>; S3 Extended Request ID:<EXT_REQ_ID>; Proxy: null), S3 Extended Request ID:<REQ_ID>
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1879)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleServiceErrorResponse(AmazonHttpClient.java:1418)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1387)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1157)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:814)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:781)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:755)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:715)
at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:697)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:561)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:541)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5456)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5403)
at com.amazonaws.services.s3.AmazonS3Client.createBucket(AmazonS3Client.java:1131)
at com.amazonaws.services.s3.AmazonS3Client.createBucket(AmazonS3Client.java:1059)

[me-ydv-5]

      var bucketRequest = new CreateBucketRequest(bucketName);

      bucketRequest.putCustomRequestHeader("Content-MD5",
          Arrays.toString(Md5Utils.computeMD5Hash(new ByteArrayInputStream(bucketRequest.toString().getBytes(
              StandardCharsets.UTF_8)))));
      s3Client.createBucket(bucketRequest);

I tried this snippet and added the content md5 hash but I get com.amazonaws.services.s3.model.AmazonS3Exception: The Content-MD5 you specified was invalid.

[me-ydv-5]

      var createBucketRequest = new CreateBucketRequest(bucketName);

      XmlWriter xml = new XmlWriter();
      if (!StringUtils.upperCase(region.getName()).equals(Region.US_Standard.toString())) {
        xml.start("CreateBucketConfiguration", "xmlns", Constants.XML_NAMESPACE);
        xml.start("LocationConstraint").value(region.getName()).end();
        xml.end();
      }

      var md5Base64 = Md5Utils.md5AsBase64(new ByteArrayInputStream(xml.getBytes()));
      createBucketRequest.putCustomRequestHeader("Content-MD5", md5Base64);
      s3Client.createBucket(createBucketRequest);

I was able to create a bucket with this code (which I copied from the sdk's createBucket method) to attach the Content-MD5 header. So looks like we just need to add the Content-MD5 header in the sdk. Happy to create PR but I'm not allowed to push code to this repo.

Think a change like this would help:

        /*
         * We can only send the CreateBucketConfiguration if we're *not*
         * creating a bucket in the US region.
         */
        if (requestRegion != null && !StringUtils.upperCase(requestRegion).equals(Region.US_Standard.toString())) {
            XmlWriter xml = new XmlWriter();
            xml.start("CreateBucketConfiguration", "xmlns", Constants.XML_NAMESPACE);
            xml.start("LocationConstraint").value(requestRegion).end();
            xml.end();

            request.setContent(new ByteArrayInputStream(xml.getBytes()));
            populateRequestHeaderWithMd5(request, xml.getBytes()); // <-- Add this line to AmazonS3Client.java::createBucket, around line 845 on current master branch
        }

[github-actions[bot]]

COMMENT VISIBILITY WARNING

Comments on closed issues are hard for our team to see. If you need more assistance, please open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.