Add support to files encrypted with KMS Server Side encryption for V2

[LeoColman]

S3 supports putting objects with KMS serverside encryption. I believe this is unsupported by S3Mock due to the following:

Given a file sender

internal class S3FileSender(
    private val s3Client: S3Client
) {

    fun send(
        bucket: String,
        sftpFile: SftpFile,
        filename: String,
        kmsKey: String? = null
    ) {
        s3Client.putObject(
            PutObjectRequest.builder().key(filename).bucket(bucket).ssekmsKeyId(kmsKey).serverSideEncryption(AWS_KMS).build(),
            RequestBody.fromInputStream(sftpFile.stream, sftpFile.contentLength)
        )
    }
}

The following piece of code fails:

s3MockExtension.registerKMSKeyRef(kmsKey)
                val client = s3MockExtension.createS3ClientV2().withBucket("bucket")
                val target = S3FileSender(client)
                target.send("bucket", sftpFile, filename, kmsKey)

With the stacktrace:

software.amazon.awssdk.core.exception.SdkClientException: Unable to unmarshall response (null). Response Code: 200, Response Text: OK

    at software.amazon.awssdk.core.exception.SdkClientException$BuilderImpl.build(SdkClientException.java:97)
    at software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.handleSuccessResponse(HandleResponseStage.java:100)
    at software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.handleResponse(HandleResponseStage.java:70)
    at software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:58)
    at software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:41)
    at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
    at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:64)
    at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:36)
    at software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:77)
    at software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:39)
    at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage$RetryExecutor.doExecute(RetryableStage.java:113)
    at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage$RetryExecutor.execute(RetryableStage.java:86)
    at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:62)
    at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:42)
    at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
    at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:57)
    at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:37)
    at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.executeWithTimer(ApiCallTimeoutTrackingStage.java:80)
    at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:60)
    at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:42)
    at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
    at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
    at software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:37)
    at software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:26)
    at software.amazon.awssdk.core.internal.http.AmazonSyncHttpClient$RequestExecutionBuilderImpl.execute(AmazonSyncHttpClient.java:240)
    at software.amazon.awssdk.core.client.handler.BaseSyncClientHandler.invoke(BaseSyncClientHandler.java:96)
    at software.amazon.awssdk.core.client.handler.BaseSyncClientHandler.execute(BaseSyncClientHandler.java:120)
    at software.amazon.awssdk.core.client.handler.BaseSyncClientHandler.execute(BaseSyncClientHandler.java:73)
    at software.amazon.awssdk.core.client.handler.SdkSyncClientHandler.execute(SdkSyncClientHandler.java:44)
    at software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler.execute(AwsSyncClientHandler.java:55)
    at software.amazon.awssdk.services.s3.DefaultS3Client.putObject(DefaultS3Client.java:3056)
    at br.com.guiabolso.sftptos3connector.internal.s3.S3FileSender.send(S3FileSender.kt:19)
    at br.com.guiabolso.sftptos3connector.internal.s3.S3FileSenderTest$1$2.invokeSuspend(S3FileSenderTest.kt:41)
    at br.com.guiabolso.sftptos3connector.internal.s3.S3FileSenderTest$1$2.invoke(S3FileSenderTest.kt)
    at io.kotlintest.specs.AbstractBehaviorSpec$GivenContext$addWhenContext$2.invokeSuspend(AbstractBehaviorSpec.kt:38)
    at io.kotlintest.specs.AbstractBehaviorSpec$GivenContext$addWhenContext$2.invoke(AbstractBehaviorSpec.kt)
    at io.kotlintest.runner.jvm.TestCaseExecutor$executeTest$supervisorJob$1$invokeSuspend$$inlined$map$lambda$1.invokeSuspend(TestCaseExecutor.kt:133)
    at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
    at kotlinx.coroutines.DispatchedTask.run(Dispatched.kt:233)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NullPointerException
    at software.amazon.awssdk.services.s3.checksums.ChecksumsEnabledValidator.validatePutObjectChecksum(ChecksumsEnabledValidator.java:129)
    at software.amazon.awssdk.services.s3.internal.handlers.SyncChecksumValidationInterceptor.afterUnmarshalling(SyncChecksumValidationInterceptor.java:90)
    at software.amazon.awssdk.core.interceptor.ExecutionInterceptorChain.lambda$afterUnmarshalling$9(ExecutionInterceptorChain.java:152)
    at software.amazon.awssdk.core.interceptor.ExecutionInterceptorChain.reverseForEach(ExecutionInterceptorChain.java:210)
    at software.amazon.awssdk.core.interceptor.ExecutionInterceptorChain.afterUnmarshalling(ExecutionInterceptorChain.java:152)
    at software.amazon.awssdk.core.client.handler.BaseClientHandler.runAfterUnmarshallingInterceptors(BaseClientHandler.java:138)
    at software.amazon.awssdk.core.client.handler.BaseClientHandler.lambda$interceptorCalling$2(BaseClientHandler.java:151)
    at software.amazon.awssdk.core.client.handler.AttachHttpMetadataResponseHandler.handle(AttachHttpMetadataResponseHandler.java:40)
    at software.amazon.awssdk.core.client.handler.AttachHttpMetadataResponseHandler.handle(AttachHttpMetadataResponseHandler.java:28)
    at software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.handleSuccessResponse(HandleResponseStage.java:89)
    ... 40 more

This is due to a checksum validation based on some headers that S3Mock doesn't return when using the encrypted endpoint, apparently.

[agudian]

Hmm, weird. We do support using KMS keys when uploading and there are tests for that using the AWS S3 Java SDK in this repo.

Can you check if you can create a breaking test case in this repository?

[LeoColman]

I think the issue is by using the new version. I'll create a breaking case.

[ChaithanyaGK]

@Kerooker There is a bug in AWS Java SDK V2 which is causing this issue. I've opened a PR https://github.com/aws/aws-sdk-java-v2/pull/1535 on SDK V2 to fix it.

[LeoColman]

I see. Thanks, @ChaithanyaGK

[afranken]

https://github.com/aws/aws-sdk-java-v2/pull/1535 has been fixed long ago, closing.