gaul / s3proxy

Access other storage backends via the S3 API
Apache License 2.0
1.65k stars 220 forks source link

Support for conditional PUT in Openstack Swift #627

Open PrimeF opened 2 months ago

PrimeF commented 2 months ago

Hi all,

according to OpenStack Swift's documentation, this object storage supports conditional PutObject requests via the optional If-None-Match header (https://docs.openstack.org/api-ref/object-store/index.html#id95).

However, when using the boto3 Python client and setting this header, it appears to have no effect since the object is overwritten. Does s3proxy handle or forward this header properly to Swift?

I was able to find some logic in s3proxy's codebase to handle conditional CopyObject operations via the x-amz-copy-source-if-none-match header but have found nothing for the PutObject operation.

Example code:

import boto3

def process_custom_arguments(params, context, **kwargs):
    if (custom_headers := params.pop("custom_headers", None)):
        context["custom_headers"] = custom_headers

def add_custom_headers(params, context, **kwargs):
    if (custom_headers := context.get("custom_headers")):
        params["headers"].update(custom_headers)
        print(params["headers"])

event_system = boto3_client.meta.events
event_system.register('before-parameter-build.s3.PutObject', process_custom_arguments)
event_system.register('before-call.s3.PutObject', add_custom_headers)

boto3_session_config = {
    "signature_version": "s3v4",
    "s3": {"addressing_style": "path"},
}
boto3_config = {
    "service_name": "s3",
    "endpoint_url": "",
    "aws_access_key_id": "",
    "aws_secret_access_key": ""}
boto3_session = boto3.session.Config(**boto3_session_config)
boto3_client = boto3.client(**boto3_config, config=boto3_session)

custom_headers = {'If-None-Match': '*', 'Expect': '100-continue'}
boto3_client.put_object(Bucket="my-bucket", Key="test_conditional_put.txt", Body=open("./test_conditional_put.txt", "rb"), custom_headers=custom_headers)