search

aws / aws-sdk-java

The official AWS SDK for Java 1.x (In Maintenance Mode, End-of-Life on 12/31/2025). The AWS SDK for Java 2.x is available here: https://github.com/aws/aws-sdk-java-v2/
https://aws.amazon.com/sdkforjava
Apache License 2.0
4.13k stars 2.83k forks source link

deleteObject ignores isRequesterPays #2862

Closed jfromme closed 4 months ago

jfromme commented 2 years ago

Describe the bug

The deleteObject methods in AmazonS3Client do not check or use the request's isRequesterPays value.

Expected Behavior

When a delete object request has isRequesterPays set to true and the target bucket has requester pays enabled then the request should succeed assuming the user has the required permsisions.

Current Behavior

Instead the request fails with an AccessDenied error.

Reproduction Steps

Attempt to delete an object from a requester pays enabled bucket.

Possible Solution

deleteObject should call the populateRequesterPaysHeader as other methods in the client do.

Additional Information/Context

No response

AWS Java SDK version used

1.11.931

JDK version used

8u212

Operating System and version

Alpine Linux 3.9

debora-ito commented 2 years ago

Are you sure the requester has the necessary permissions to access the object? AccessDenied indicates it has not. Are you providing the requesterPays parameter in DeleteObjectRequest? Can you share a repro code?

jfromme commented 2 years ago

I believe the permissions are okay. Switching from using deleteObject() to deleteObjects() worked without any modifications to the user or bucket permissions.

Here is a test case which uses MockServer to set expectations on the request being sent. testDeleteObject() fails with a 404 because the expected request header is absent. In contrast, testDeleteObjects() passes. https://github.com/jfromme/delete-object-test

Here is the MockServer log from testDeleteObject():

2022-10-10 16:27:40 5.14.0 INFO 52920 started on port: 52920 
2022-10-10 16:27:41 5.14.0 INFO 52920 creating expectation:

  {
    "httpRequest" : {
      "method" : "DELETE",
      "path" : "/test-bucket/file-key",
      "headers" : {
        "x-amz-request-payer" : [ "requester" ]
      }
    },
    "httpResponse" : {
      "statusCode" : 204
    },
    "id" : "3555c98f-28b1-4216-8d62-68f13223e655",
    "priority" : 0,
    "timeToLive" : {
      "unlimited" : true
    },
    "times" : {
      "unlimited" : true
    }
  }

 with id:

  3555c98f-28b1-4216-8d62-68f13223e655

2022-10-10 16:27:41 5.14.0 INFO 52920 received request:

  {
    "method" : "DELETE",
    "path" : "/test-bucket/file-key",
    "headers" : {
      "x-amz-content-sha256" : [ "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" ],
      "content-length" : [ "0" ],
      "content-encoding" : [ ".*" ],
      "amz-sdk-retry" : [ "0/0/500" ],
      "amz-sdk-request" : [ "attempt=1;max=4" ],
      "amz-sdk-invocation-id" : [ "ae9d22c7-80eb-f7f7-99e0-0e6df6702135" ],
      "X-Amz-Date" : [ "20221010T202741Z" ],
      "User-Agent" : [ "aws-sdk-java/1.12.319 Mac_OS_X/10.15.7 OpenJDK_64-Bit_Server_VM/25.312-b07 java/1.8.0_312 vendor/Amazon.com_Inc. cfg/retry-mode/legacy" ],
      "Host" : [ "localhost:52920" ],
      "Content-Type" : [ "application/octet-stream" ],
      "Connection" : [ "Keep-Alive" ],
      "Authorization" : [ "AWS4-HMAC-SHA256 Credential=AKIA5SOSPFND4TX4E4Y7/20221010/us-east-1/s3/aws4_request, SignedHeaders=amz-sdk-invocation-id;amz-sdk-request;amz-sdk-retry;content-type;host;user-agent;x-amz-content-sha256;x-amz-date, Signature=b8d6c98498f9bc24ff46a4022826363e914d595f234aeecfb2d08ab77b800cdc" ]
    },
    "keepAlive" : true,
    "secure" : false,
    "localAddress" : "127.0.0.1:52920",
    "remoteAddress" : "127.0.0.1"
  }

2022-10-10 16:27:41 5.14.0 INFO 52920 request:

  {
    "method" : "DELETE",
    "path" : "/test-bucket/file-key",
    "headers" : {
      "x-amz-content-sha256" : [ "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" ],
      "content-length" : [ "0" ],
      "content-encoding" : [ ".*" ],
      "amz-sdk-retry" : [ "0/0/500" ],
      "amz-sdk-request" : [ "attempt=1;max=4" ],
      "amz-sdk-invocation-id" : [ "ae9d22c7-80eb-f7f7-99e0-0e6df6702135" ],
      "X-Amz-Date" : [ "20221010T202741Z" ],
      "User-Agent" : [ "aws-sdk-java/1.12.319 Mac_OS_X/10.15.7 OpenJDK_64-Bit_Server_VM/25.312-b07 java/1.8.0_312 vendor/Amazon.com_Inc. cfg/retry-mode/legacy" ],
      "Host" : [ "localhost:52920" ],
      "Content-Type" : [ "application/octet-stream" ],
      "Connection" : [ "Keep-Alive" ],
      "Authorization" : [ "AWS4-HMAC-SHA256 Credential=AKIA5SOSPFND4TX4E4Y7/20221010/us-east-1/s3/aws4_request, SignedHeaders=amz-sdk-invocation-id;amz-sdk-request;amz-sdk-retry;content-type;host;user-agent;x-amz-content-sha256;x-amz-date, Signature=b8d6c98498f9bc24ff46a4022826363e914d595f234aeecfb2d08ab77b800cdc" ]
    },
    "keepAlive" : true,
    "secure" : false,
    "localAddress" : "127.0.0.1:52920",
    "remoteAddress" : "127.0.0.1"
  }

 didn't match expectation:

  {
    "httpRequest" : {
      "method" : "DELETE",
      "path" : "/test-bucket/file-key",
      "headers" : {
        "x-amz-request-payer" : [ "requester" ]
      }
    },
    "httpResponse" : {
      "statusCode" : 204
    },
    "id" : "3555c98f-28b1-4216-8d62-68f13223e655",
    "priority" : 0,
    "timeToLive" : {
      "unlimited" : true
    },
    "times" : {
      "unlimited" : true
    }
  }

 because:

  method matched
  path matched
  body matched
  headers didn't match

2022-10-10 16:27:41 5.14.0 INFO 52920 no expectation for:

  {
    "method" : "DELETE",
    "path" : "/test-bucket/file-key",
    "headers" : {
      "x-amz-content-sha256" : [ "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" ],
      "content-length" : [ "0" ],
      "content-encoding" : [ ".*" ],
      "amz-sdk-retry" : [ "0/0/500" ],
      "amz-sdk-request" : [ "attempt=1;max=4" ],
      "amz-sdk-invocation-id" : [ "ae9d22c7-80eb-f7f7-99e0-0e6df6702135" ],
      "X-Amz-Date" : [ "20221010T202741Z" ],
      "User-Agent" : [ "aws-sdk-java/1.12.319 Mac_OS_X/10.15.7 OpenJDK_64-Bit_Server_VM/25.312-b07 java/1.8.0_312 vendor/Amazon.com_Inc. cfg/retry-mode/legacy" ],
      "Host" : [ "localhost:52920" ],
      "Content-Type" : [ "application/octet-stream" ],
      "Connection" : [ "Keep-Alive" ],
      "Authorization" : [ "AWS4-HMAC-SHA256 Credential=AKIA5SOSPFND4TX4E4Y7/20221010/us-east-1/s3/aws4_request, SignedHeaders=amz-sdk-invocation-id;amz-sdk-request;amz-sdk-retry;content-type;host;user-agent;x-amz-content-sha256;x-amz-date, Signature=b8d6c98498f9bc24ff46a4022826363e914d595f234aeecfb2d08ab77b800cdc" ]
    },
    "keepAlive" : true,
    "secure" : false,
    "localAddress" : "127.0.0.1:52920",
    "remoteAddress" : "127.0.0.1"
  }

 returning response:

  {
    "statusCode" : 404,
    "reasonPhrase" : "Not Found"
  }
debora-ito commented 1 year ago

@jfromme apologies for the long silence.

Yes, I can reproduce, the x-amz-requester-payer is not being added to the request. The workaround is to use DeleteObjects.

debora-ito commented 4 months ago

Hi @jfromme

We apologize but this won't get fixed in v1 before going into Maintenance Mode. The workaround is to use DeleteObjects (plural) if you need to enable isRequesterPays.

Reference:

github-actions[bot] commented 4 months ago

This issue is now closed.

Comments on closed issues are hard for our team to see. If you need more assistance, please open a new issue that references this one.