(short issue description)Adding any value in the checksumSHA256() of PutObjectRequest is allowing the object to upload.

[lokeshvikram]

Describe the bug

Hi, I am facing issues in upload of object via s3AsyncClient in single part as well as multipart uploads when adding .checksumAlgorithm() as well as .checksumSHA256() in my put object request as well as UploadPartRequest.

1. When I don't add any of the above two mentioned params and try upload, getting this error S3Exception: the content-sha256 you specified did not match what we received.

while inspecting logs, I am seeing x-amz-content-sha256 value as STREAMING-UNSIGNED-PAYLOAD-TRAILER and x-amz-trailer:x-amz-checksum-crc32 why it is coming as streaming unsigned payload trailer? not able to get this.

But when I add this parameter in put object request -> .checksumSHA256() with value passed as any string other than empty string, then the upload is happening. In this instance, the headers in the logs were:

x-amz-checksum-sha256: ab (ab was the string i used in the .checksumSHA256()) x-amz-content-sha256: UNSIGNED-PAYLOAD

but no x-amz-trailer header is present here.

Now when I add .checksumAlgorithm(ChecksumAlgorithm.SHA256), then upload is passing and i am able to see these headers: x-amz-checksum-sha256: ab (ab was the string i used in the .checksumSHA256()) x-amz-content-sha256: UNSIGNED-PAYLOAD x-amz-sdk-checksum-algorithm: SHA256

whenever the x-amz-content-sha256 has STREAMING-UNSIGNED-PAYLOAD-TRAILER, the upload is failing.

Even in multipart request done via S3AsyncClient's automatic multipart upload or even with s3 transfer manager, when adding .checksumAlgorithm() as well as .checksumSHA256() are added, upload fails.

but when only .checksumSHA256() is used with any random string other than "" empty string, the upload passes.

when I add .checksumAlgorithm() in multipart upload, i am seeing the x-amz-content-sha256 as STREAMING-UNSIGNED-PAYLOAD-TRAILER and x-amz-trailer:x-amz-checksum-SHA256.

https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-streaming-trailers.html -> in this documentation i see, while signing chunked objects which includes trailing headers, we should pass an extra chunk object with 0 bytes. but I am not sure is this the reason or someting else?

But the inherent s3AsyncClient's multipart upload and s3TransferManager's multipart upload are transferring n chunks only and the last chunk with 0 bytes is not being passed and it fails with the above mentioned error: S3Exception: the content-sha256 you specified did not match what we received.

but the same when i remove .checksumAlgorithm() form the request and upload it is working properly.

Also the inbuild validation of checksum when passed via .checksumSHA256() is not happening not sure why?

Also unable to figure out the differences when x-amz-content-sha256 is passed as streaming-unsigned-payload-trailer, the upload is failing with the above mentioned exception.

Regression Issue

• [ ] Select this option if this issue appears to be a regression.

Expected Behavior

Upload should happen properly or need to understand why this discrepancy is there.

Current Behavior

I wont be able to include logs. the only erros are sha256 conetent doesnt match and it is related to the headers in put request

Reproduction Steps

Multipart upload using s3 transfer manager:

public void multipartUploadWithTransferManager(String filePath) {

    final SdkAsyncHttpClient httpClient = NettyNioAsyncHttpClient.builder()
            .maxConcurrency(50)
            .connectionTimeout(Duration.ofSeconds(60))
            .build();

    final S3AsyncClient s3AsyncClient = S3AsyncClient.builder().multipartEnabled(true)
            .multipartConfiguration(MultipartConfiguration.builder()
                    .thresholdInBytes((long) 8 * 1024 * 1024)
                    .minimumPartSizeInBytes((long) 8* 1024 * 1024)
                    .build())
            .region(Region.US_EAST_1)
            .endpointOverride(URI.create("ENDPOINT"))
            .httpClient(httpClient)
            .credentialsProvider(StaticCredentialsProvider.create(AwsBasicCredentials.create("accesskey", "secretkey")))
            .build();

    try(S3TransferManager transferManager = S3TransferManager.builder().
            s3Client(s3AsyncClient)
            .build())
    {
        UploadRequest uploadFileRequest = UploadRequest.builder()
                .putObjectRequest(b -> b
                        .bucket(bucketName)
                        .key(key)
                        .checksumAlgorithm(ChecksumAlgorithm.SHA256)
                        .checksumSHA256("some random string")
                )
                .requestBody(AsyncRequestBody.fromFile(Path.of(filePath)))
                .addTransferListener(LoggingTransferListener.create())
                .build();
        Upload fileUpload = transferManager.upload(uploadFileRequest);
        fileUpload.completionFuture().join();

    }
}

similarly you can use the configured s3AsyncClient to directly do a put object request and it does a multipart upload only. and event single part upload also has this problem.

Possible Solution

Not able to think of a solution.

Additional Information/Context

No response

AWS Java SDK version used

sdk bom version 2.29.2

JDK version used

jdk 17.0.9

Operating System and version

windows 11

[lokeshvikram]

I am not using aws s3, but using dell ecs object storage which is s3 compatible

[debora-ito]

@lokeshvikram before we dig deeper, can you confirm if you see the same behavior when aws s3 is used?

This is to rule out that the issue is on the Dell ECS Storage side.

[github-actions[bot]]

It looks like this issue has not been active for more than five days. In the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please add a comment to prevent automatic closure, or if the issue is already closed please feel free to reopen it.