Can't get s3 bucket object when using SSE [minio] [s3 object] [encryption]

odysseu commented 10 months ago

Hi, I am trying to copy to and get from minio s3's bucket a file encrypted with sse-c. But I can't find a way to use paws' configs, can anyone help ?

Works in python

When I use boto3 in python, it works fine :

endpoint_url='https://my.endpoint.fr'
aws_access_key_id='myaccesskey'
aws_secret_access_key='mysecretkey'
bucket_name = 'mybucketname'
object_key = 'myobject/key.txt'
encryption_key = 'thekeyiused32bits'
sse_c_algorithm = 'AES256'

import boto3

s3 = boto3.client('s3',
                  endpoint_url=endpoint_url,
                  aws_access_key_id=aws_access_key_id,
                  aws_secret_access_key=aws_secret_access_key,
                  config=boto3.session.Config(signature_version='s3v4'),
                  #addressing_style='path',
                  region_name='',
                  use_ssl=True)

response = s3.get_object(Bucket=bucket_name, Key=object_key, SSECustomerKey=encryption_key, SSECustomerAlgorithm=sse_c_algorithm)

content = response['Body'].read()
print(content.decode('utf-8'))

which ouputs the content of the file 👍

Does not work in R

From @DyfanJones in https://github.com/cloudyr/aws.s3/issues/433#issuecomment-1805612553 I understand it should be pretty easy to implement the same in R with paws but here's what I get :

library(paws)

Sys.setenv("AWS_ACCESS_KEY_ID" = "...",
           "AWS_SECRET_ACCESS_KEY" = "...",
           "AWS_DEFAULT_REGION" = "us-east-1",
           "AWS_S3_ENDPOINT"= "my.endpoint.fr")

access_key <- "..."
secret_key <- "..."
bucket_name <- "projet-test"
object_key <- "myobject/key.txt"
sse_c_key <- "..."
ssec_algorithm <- "AES256"
### 

minio <- paws::s3(
  config = list(
    credentials = list(
      creds = list(
        access_key_id = Sys.getenv("AWS_ACCESS_KEY_ID"),
        secret_access_key = Sys.getenv("AWS_SECRET_ACCESS_KEY"),
        session_token = Sys.getenv("AWS_SESSION_TOKEN")
        )
      ),
    endpoint = paste0("https://", Sys.getenv("AWS_S3_ENDPOINT")),
    region = Sys.getenv("AWS_DEFAULT_REGION")
    )
  ) # OK

minio$list_buckets() # OK

I can't find the way to use the SSE options :

minio$get_object(SSECustomerAlgorithm = "AES256",
                 Bucket="mybucketname", 
                 Key = "myobject/key.txt",
                 SSECustomerKey = paste0("projet-test/myobject/=",sse_c_key)
                 )
# Error in file(what, "rb") : cannot open the connection
# In addition: Warning message:
#   In file(what, "rb") :
#   cannot open file 'projet-test/open_data/=...sseckey...': No such file or directory

But also tried whithout the bucket file path :

```R minio$get_object(SSECustomerAlgorithm = "AES256", Bucket="mybucketname", Key = "myobject/key.txt", SSECustomerKey = sse_c_key) ) # Error in file(what, "rb") : cannot open the connection # In addition: Warning message: # In file(what, "rb") : # cannot open file 'sse_c_key': No such file or directory ```

However I get another error which I also don't understand when I use the sse key written in a local file localfile/key , and try to put a file myobject/key.txt instead of getting it :


minio$put_object("localfile/hello.txt",
                 Bucket = "mybucketname",
                 Key = "myobject/key.txt",
                 SSECustomerAlgorithm = "AES256",
                 SSECustomerKey = "localfile/key")
# Error: InvalidArgument (HTTP 400). Requests specifying Server Side Encryption with 
# Customer provided keys must provide the client calculated MD5 of the secret key.

would appreciate help :)