s3 InvalidBucketAclWithObjectOwnership error when trying to disable ACLs

pivotal-marcela-campo commented 2 years ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform AWS Provider Version

Terraform v1.2.4 on darwin_amd64

provider registry.terraform.io/hashicorp/aws v4.25.0

Affected Resource(s)

aws_s3_bucket_ownership_controls
aws_s3_bucket_acl

Terraform Configuration Files

Please include all Terraform configurations required to reproduce the bug. Bug reports without a functional reproduction may be closed without investigation.

variable acl { type = string }
variable ownership { type = string }

resource "aws_s3_bucket" "b" {
  bucket = "csb-8938b4c0-d67f-4c34-9f68-a66deef99b3d"

}

resource "aws_s3_bucket_ownership_controls" "disallow_acls" {
  bucket = aws_s3_bucket.b.id

  rule {
    object_ownership = var.ownership
  }
}

resource "aws_s3_bucket_acl" "example_bucket_acl" {
  count  = length(var.acl) != 0 ? 1 : 0
  bucket = aws_s3_bucket.b.id
  acl    = var.acl

}

provider "aws" {
  version = "4.25.0"
  region  = "us-west-2"
  access_key = "..."
  secret_key = "..."
}

Debug Output

https://gist.github.com/pivotal-marcela-campo/84feafdef53f9cb8a92f2434b9b20816

Panic Output

Expected Behavior

ACLs should have been disabled with ownership set to BucketOwnerEnforced. Plan shows this is what will happen (1 resource deletion and 1 modification in place) but it is not the end state.

Actual Behavior

Got error

022-08-08T14:30:40.419+0100 [ERROR] vertex "aws_s3_bucket_ownership_controls.disallow_acls" error: error updating S3 Bucket (csb-8938b4c0-d67f-4c34-9f68-a66deef99b3d) Ownership Controls: InvalidBucketAclWithObjectOwnership: Bucket cannot have ACLs set with ObjectOwnership's BucketOwnerEnforced setting
    status code: 400, request id: KMSJ70AXJF116GVP, host id: A5yoOrRmJYNRiUyIEQjaOu6HflYmMK1NkUN8bY8InoYmI1FL9vrSpSjab/4PS9SNZyZEOcbzPdU=
╷
│ Error: error updating S3 Bucket (csb-8938b4c0-d67f-4c34-9f68-a66deef99b3d) Ownership Controls: InvalidBucketAclWithObjectOwnership: Bucket cannot have ACLs set with ObjectOwnership's BucketOwnerEnforced setting
│   status code: 400, request id: KMSJ70AXJF116GVP, host id: A5yoOrRmJYNRiUyIEQjaOu6HflYmMK1NkUN8bY8InoYmI1FL9vrSpSjab/4PS9SNZyZEOcbzPdU=
│
│   with aws_s3_bucket_ownership_controls.disallow_acls,
│   on main.tf line 52, in resource "aws_s3_bucket_ownership_controls" "disallow_acls":
│   52: resource "aws_s3_bucket_ownership_controls" "disallow_acls" {

output of tf plan looks fine


mc  ○ → tf apply -var=acl= -var=ownership=BucketOwnerEnforced
aws_s3_bucket.b: Refreshing state... [id=csb-8938b4c0-d67f-4c34-9f68-a66deef99b3d]
aws_s3_bucket_ownership_controls.disallow_acls: Refreshing state... [id=csb-8938b4c0-d67f-4c34-9f68-a66deef99b3d]
aws_s3_bucket_acl.example_bucket_acl[0]: Refreshing state... [id=csb-8938b4c0-d67f-4c34-9f68-a66deef99b3d,public-read]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  ~ update in-place
  - destroy

Terraform will perform the following actions:

  # aws_s3_bucket_acl.example_bucket_acl[0] will be destroyed
  # (because index [0] is out of range for count)
  - resource "aws_s3_bucket_acl" "example_bucket_acl" {
      - acl    = "public-read" -> null
      - bucket = "csb-8938b4c0-d67f-4c34-9f68-a66deef99b3d" -> null
      - id     = "csb-8938b4c0-d67f-4c34-9f68-a66deef99b3d,public-read" -> null

      - access_control_policy {
          - grant {
              - permission = "READ" -> null

              - grantee {
                  - type = "Group" -> null
                  - uri  = "http://acs.amazonaws.com/groups/global/AllUsers" -> null
                }
            }
          - grant {
              - permission = "FULL_CONTROL" -> null

              - grantee {
                  - display_name = "pcf-services-enablement" -> null
                  - id           = "bdc2ddf8fd7c88c34c94f8199df69fbe2d929591b3862aa6ef632451c3a0e2f5" -> null
                  - type         = "CanonicalUser" -> null
                }
            }

          - owner {
              - display_name = "pcf-services-enablement" -> null
              - id           = "bdc2ddf8fd7c88c34c94f8199df69fbe2d929591b3862aa6ef632451c3a0e2f5" -> null
            }
        }
    }

  # aws_s3_bucket_ownership_controls.disallow_acls will be updated in-place
  ~ resource "aws_s3_bucket_ownership_controls" "disallow_acls" {
        id     = "csb-8938b4c0-d67f-4c34-9f68-a66deef99b3d"
        # (1 unchanged attribute hidden)

      ~ rule {
          ~ object_ownership = "ObjectWriter" -> "BucketOwnerEnforced"
        }
    }

Plan: 0 to add, 1 to change, 1 to destroy.

Steps to Reproduce

terraform apply with acl=public-read and ownership=ObjectWriter
terraform apply with acl=. (no acl) and ownership=BucketOwnerEnforced

I also tried setting acl to either private or null in the resource instead of using count, but the end result is the same.

It also breaks in the same fashion when trying to enable ACLs on a bucket that has them disabled.

Important Factoids

References

0000

acdha commented 2 years ago

I was able to resolve this by setting the bucket ACL to private and clearing additional ACLs which had been set. I definitely think this experience could be smoother but I wonder whether that might be tricky to do better than by extending the documentation.

devbyaccident commented 2 years ago

I was able to resolve this by setting the bucket ACL to private and clearing additional ACLs which had been set.

Confirming that this works on a bucket-by-bucket basis, but not an ideal solution for orgs managing a lot of different buckets.

pivotal-marcela-campo commented 2 years ago

Agree with @devbyaccident here, manual intervention is not possible for our use case.

bill-scalapay commented 1 year ago

Perhaps we could add an argument to the aws_s3_bucket_ownership_controls resource such as force_destroy_acls that replaces any existing ACLs for the bucket with the private canned ACL before trying to set the object ownership to BucketOwnerEnforced?

I ran into this issue when trying to write a child module for an S3 bucket. It included a variable enable_cloudfront_logging which would create the required ACL for allowing a CloudFront distribution to log to the bucket. For the best user experience, I wanted it so that toggling the variable either way and applying the Terraform just worked, with no manual steps required.

I thought I'd found a crude solution by having two aws_s3_bucket_acl resources, one with the private canned ACL and one with the CloudFront ACL. Using depends_on and count, I'd set it up so that the private canned ACL would be applied before setting the object ownership to BucketOwnerEnforced, and setting the object ownership to BucketOwnerPreferred would happen before applying the CloudFront ACL. However, I realised this doesn't work when creating a new bucket that doesn't have CloudFront logging enabled, as by default the new bucket will already have the BucketOwnerEnforced setting, so the private canned ACL can't be applied.

It seems that the problem is that the S3 API is very particular about the ordering of these changes. However, implementing this ordering effectively in Terraform is a headache, if even possible. Having a way to overwrite the ACLs from the aws_s3_bucket_ownership_controls resource would make everything simpler, and I think there's some precedent here considering that the aws_s3_bucket resource has the force_destroy argument that first deletes all the objects in the bucket.

fclesio commented 1 year ago

Just another datapoint: The official Associate Tutorial from Hashicorp has the same issue.

prashant0085 commented 1 year ago

Also faced the same issue today, Tried to disable s3 acl by changing the bucket owner to BucketOwnerEnforced for bucket which already had s3-log-delivery READ and WRITE access and got below error:

Error: creating S3 Bucket (<bucket-name> Ownership Controls: InvalidBucketAclWithObjectOwnership: Bucket cannot have ACLs set with ObjectOwnership's BucketOwnerEnforced setting
│ 
│   with module.s3_logging[0].aws_s3_bucket_ownership_controls.change-ownership-s3-disable-acl[0],
│   on .terraform/modules/s3_logging/s3/main.tf line 175, in resource "aws_s3_bucket_ownership_controls" "change-ownership-s3-disable-acl":
│  175: resource "aws_s3_bucket_ownership_controls" "change-ownership-s3-disable-acl" {
│

prashant0085 commented 1 year ago

I was able to resolve this by setting the bucket ACL to private and clearing additional ACLs which had been set. I definitely think this experience could be smoother but I wonder whether that might be tricky to do better than by extending the documentation.

@acdha Did you did the above change manually to fix it?

modwyer42 commented 12 months ago

@prashant0085 I had a similar requirement of transitioning S3 access logging buckets to use bucket policy and have ACLs disabled (BucketOwnerEnforced). I can do it via Terraform, but it's a multi-step process, which is not scalable:

Update code to set ACLs to private and leave bucket ownership unchanged. (Ensure you also apply the required bucket policy configuration, otherwise access logging stops)
Apply these updates to the bucket. This essentially sets the ACLs to the "old default" which AWS/the provider then seems OK with disabling.
Update the code to set the ACL to null (or remove it from code) and set bucket ownership to BucketOwnerEnforced.
Apply these changes to the bucket.

I figure a lot of the problem we're having here is the "feature" of the provider where it doesn't actually modify the ACL when you delete the resource. It just deletes it from state: From the provider docs

Even if the provider just "reset" the ACL to a base/default version it would be more useful, and may resolve this issue.

sparr commented 7 months ago

I am encountering this problem when creating buckets using http://github.com/samstav/terraform-aws-backend

Error: creating S3 Bucket (foo-tf-state-logs): operation error S3: CreateBucket, https response error StatusCode: 400, RequestID: XKSF4PSZPC0TFSRK, HostID: RwFULMMW2Lvj/CXLARoI9m1YZup/gYgM6+SToJn+T/3zIaWxpHGTYfVIomoJRY+XMMoYmPDa6VfZgFyezUvQBA==, api error InvalidBucketAclWithObjectOwnership: Bucket cannot have ACLs set with ObjectOwnership's BucketOwnerEnforced setting

resource "aws_s3_bucket" "tf_backend_logs_bucket" {
  bucket = "${var.backend_bucket}-logs"
  acl    = "log-delivery-write"
  versioning {
    enabled = true
  }
}

rmccarthy-ellevation commented 5 months ago

Any update on this?

hashicorp / terraform-provider-aws