boto / s3transfer

Amazon S3 Transfer Manager for Python
Apache License 2.0
209 stars 133 forks source link

TransferManager.copy uses ExpectedBucketOwner in head_object request on source object #232

Open pierre-muller opened 2 years ago

pierre-muller commented 2 years ago

The copy operations executes a head_object request on the source object, and uses the ExpectedBucketOwner argument in this request. This results in Forbidden errors when source and destination buckets are owned by different owners.

>>> client.get_bucket_acl(Bucket='bucket-from')['Owner']['ID']
'50cea555-eac6-4373-bc27-7333bfe6abe0:50cea555-eac6-4373-bc27-7333bfe6abe0'
>>> client_to.get_bucket_acl(Bucket='bucket-to')['Owner']['ID']
'88e5206f-48b0-4cb6-b2db-fc26be02031d:88e5206f-48b0-4cb6-b2db-fc26be02031d'
>> source = {'Bucket': 'bucket-from', 'Key': 'foo'}
>>> client.head_object(**source, ExpectedBucketOwner= '50cea555-eac6-4373-bc27-7333bfe6abe0')['ContentLength']
3200000000
>>> client.copy(source, 'bucket-to', 'bar', ExtraArgs={'ExpectedBucketOwner': '88e5206f-48b0-4cb6-b2db-fc26be02031d'})
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.9/site-packages/boto3/s3/inject.py", line 444, in copy
    return future.result()
  File "/usr/local/lib/python3.9/site-packages/s3transfer/futures.py", line 103, in result
    return self._coordinator.result()
  File "/usr/local/lib/python3.9/site-packages/s3transfer/futures.py", line 266, in result
    raise self._exception
  File "/usr/local/lib/python3.9/site-packages/s3transfer/tasks.py", line 269, in _main
    self._submit(transfer_future=transfer_future, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/s3transfer/copies.py", line 118, in _submit
    response = call_args.source_client.head_object(
  File "/usr/local/lib/python3.9/site-packages/botocore/client.py", line 391, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/local/lib/python3.9/site-packages/botocore/client.py", line 719, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (403) when calling the HeadObject operation: Forbidden
>>> client.put_object(Bucket='bucket-to', Key='bar', ExpectedBucketOwner='88e5206f-48b0-4cb6-b2db-fc26be02031d')['ETag']
'"d41d8cd98f00b204e9800998ecf8427e"'
pierre-muller commented 2 years ago

tested with version: s3transfer 0.5.1 boto3 1.21.6 botocore 1.24.6