aws / aws-sdk-java

The official AWS SDK for Java 1.x. The AWS SDK for Java 2.x is available here: https://github.com/aws/aws-sdk-java-v2/
https://aws.amazon.com/sdkforjava
Apache License 2.0
4.12k stars 2.83k forks source link

[S3] Cant find way to set encryption-context with AmazonS3EncryptionClientBuilder #1631

Closed sunilsp54 closed 4 years ago

sunilsp54 commented 6 years ago

I am creating AmazonS3Encryption client from AmazonS3EncryptionClientBuilder. I need to set encryption context for each s3 object. I could not find anything about this in documentation. This is for Client Side encryption with KMS managed key. Please help.

varunnvs92 commented 6 years ago

What do you mean by setting encryption context for each s3 object? Can you give an example of sample encryption context you need to set?

sunilsp54 commented 6 years ago

My requirements is , every file stored in S3 needs to use "fileId" for encyption and decryption. "fileId" as key value pair in encryption-context can be used to achieve this. without the "fileId" the file would not be decryptable. So for decrypting the file, master key id from Kms and fileId are required.

varunnvs92 commented 6 years ago

Can you share some docs on this field? Or is this a custom thing for about application?

sunilsp54 commented 6 years ago

Yes it is custom field of type String from the application. Ignoring my use case, can you show me some code to set encryption-context using AmazonS3Encryption client.

There is some information here

But how to add my own encryption context key value pairs additional to the ones shown in the above link.

We can set encryption-context with encryption sdk as follows

final EncryptRequest enc = new EncryptRequest(); enc.setEncryptionContext(Collections.singletonMap("FILE-ID", "sdadsasdafyyrtruirooo")); final ByteBuffer ciphertext = kms.encrypt(enc).getCiphertextBlob(); How to do the same with AmazonS3Encryption client ?

varunnvs92 commented 6 years ago

This code might be relavant to your use case https://github.com/aws/aws-sdk-java/blob/4734de6fb0f80fe5768a6587aad3b9d0eaec388f/aws-java-sdk-s3/src/main/java/com/amazonaws/services/s3/internal/crypto/S3CryptoModuleBase.java#L529-L530 https://github.com/aws/aws-sdk-java/blob/4734de6fb0f80fe5768a6587aad3b9d0eaec388f/aws-java-sdk-s3/src/main/java/com/amazonaws/services/s3/internal/crypto/ContentCryptoMaterial.java#L900-L915

I would say set this key-value pair using EncryptionMaterials#addDescription. Let us know if it worked.

sunilsp54 commented 6 years ago

I have already tried adding description as you mentioned, these descriptions are getting stored as object meta-data in s3. But I was able to download and decrypt the file without giving the same descriptions. Looking at the code here I think s3 object-meta-data from s3 are automatically being read by the s3Encryption client and are being used for decryption when s3Encryption.getObject is called.
Ideally if descriptions given while building s3Encryption client does not match that of the s3-object being requested, the decryption should fail right.

shorea commented 6 years ago

@sunilsp54 as far as I can tell this is not a feature we support. I will mark this as a feature request but it's unlikely we'll get to it anytime soon or perhaps at all within the lifetime of V1 of the SDK. If more customers +1 the issue we will include it for prioritization.

MadhuraMath commented 6 years ago

Hi, We are using Client Side Encryption to upload and download S3 objects from AWS S3 server. Below is the code snippet we have used to create encryption client object passing the cmkid. We are facing the below issue while downloading: While downloading, even when incorrect cmkid (not the same cmkid used during encryption- upload) is passed to encryption client, its getting downloaded successfully. There is no option to pass kmsid in getObject API of encryptionclient. ideally it should fail as incorrect cmkid is passed. I guess internally, when getObject is called, the encrypted datakey which is a part of object meta data, consist cmkid and KMS server decrypts it without passing cmkid again during download. We are having the concern here regarding the security. We are not understanding the part where it says if you lose cmkid you won't be able to decrypt the S3 object data which is encrypted.

Can you pl. help us here? Do we have a way to pass kmsid again from client and making sure that it doesn't get decrypt when we pass incorrect cmkid during download. Currently, its not considering the cmkid passed from client (during encryption client object creation) at all during download

// Create the encryption client. KMSEncryptionMaterialsProvider materialProvider = new KMSEncryptionMaterialsProvider(kms_cmk_id); CryptoConfiguration cryptoConfig = new CryptoConfiguration() .withAwsKmsRegion(RegionUtils.getRegion(clientRegion)); AmazonS3Encryption encryptionClient = AmazonS3EncryptionClientBuilder.standard() .withCredentials(new ProfileCredentialsProvider()) .withEncryptionMaterials(materialProvider) .withCryptoConfiguration(cryptoConfig) .withRegion(clientRegion).build();

// Download the object. The downloaded object is still encrypted. S3Object downloadedObject = encryptionClient.getObject(bucketName, keyName); S3ObjectInputStream input = downloadedObject.getObjectContent();

johnwalker commented 5 years ago

Hi @MadhuraMath , there should be a way to do this. Have you tried using the EncryptedGetObjectRequest?

/**
 * <p>
 * An extension of {@link GetObjectRequest} to allow additional encryption
 * material description to be specified on a per-request basis. In particular,
 * {@link EncryptedGetObjectRequest} is only recognized by
 * {@link AmazonS3EncryptionClient}.
 * </p>
 * <p>
 * If {@link EncryptedGetObjectRequest} is used against the non-encrypting
 * {@link AmazonS3Client}, the additional attributes will be ignored.
 * </p>
 * <p>
 * The additional material description must not conflict with the existing one
 * saved in S3 or else will cause the get request to fail fast later on.
 */
public class EncryptedGetObjectRequest extends GetObjectRequest implements Serializable {
millems commented 4 years ago

Unfortunately we won't be adding any new features like this in 1.11.x of the SDK, because we're concentrating our effort in 2.x of the SDK.