s3: can't enable bucket public access

pahud commented 1 year ago

Describe the bug

https://github.com/aws/aws-cdk/issues/25358 introduced the major changes for S3 in April 2023 but it's still unclear to customers how to setup a S3 bucket with public access enabled for some use cases like static website hosting.

Expected Behavior

As https://github.com/aws/aws-cdk/issues/25358#issuecomment-1534455073 suggested, this should work:

const bucket = new s3.Bucket(this, 'Bucket', {
    publicReadAccess: true,
    blockPublicAccess: {
        blockPublicPolicy: false,
        blockPublicAcls: false,
        ignorePublicAcls: false,
        restrictPublicBuckets: false,
    },
    accessControl: BucketAccessControl.PUBLIC_READ,
    objectOwnership: ObjectOwnership.OBJECT_WRITER,
})

Current Behavior

It would fail with this error but sometimes it deploys successfully.

I guess this could be a bug from cloudformation as it does not always fail.

12:05:26 PM | CREATE_FAILED        | AWS::S3::Bucket       | Bucket83908E77
Bucket cannot have public ACLs set with BlockPublicAccess enabled (Service: Amazon S3; Status Code: 400; Error
Code: InvalidBucketAclWithBlockPublicAccessError; Request ID: 8R35HKMW941ZRN30; S3 Extended Request ID: ZK3daYi
1wkLTk++u+/3mvRPWXBbDNstauIDnp8kiL4XdQfdmzJ2jAktdUVBpRztwEumIJteAh+8=; Proxy: null)

Another alternative is to deploy with accessControl: BucketAccessControl.PUBLIC_READ commented off.

const bucket = new s3.Bucket(this, 'Bucket', {
    publicReadAccess: true,
    blockPublicAccess: {
        blockPublicPolicy: false,
        blockPublicAcls: false,
        ignorePublicAcls: false,
        restrictPublicBuckets: false,
    },
    // accessControl: BucketAccessControl.PUBLIC_READ,
    objectOwnership: ObjectOwnership.OBJECT_WRITER,
});

And re-deploy with accessControl enabled. This will 100% work.

const bucket = new s3.Bucket(this, 'Bucket', {
    publicReadAccess: true,
    blockPublicAccess: {
        blockPublicPolicy: false,
        blockPublicAcls: false,
        ignorePublicAcls: false,
        restrictPublicBuckets: false,
    },
    accessControl: BucketAccessControl.PUBLIC_READ,
    objectOwnership: ObjectOwnership.OBJECT_WRITER,
});

I guess the cloudformation handler probably can't handle this well when both accessControl and objectOwnership are enabled.

Reproduction Steps

See current behavior.

Possible Solution

See current behavior. This might be a CFN bug.

Additional Information/Context

The synth output for the Bucket resource

Resources:
  Bucket83908E77:
    Type: AWS::S3::Bucket
    Properties:
      AccessControl: PublicRead
      OwnershipControls:
        Rules:
          - ObjectOwnership: ObjectWriter
      PublicAccessBlockConfiguration:
        BlockPublicAcls: false
        BlockPublicPolicy: false
        IgnorePublicAcls: false
        RestrictPublicBuckets: false
    UpdateReplacePolicy: Retain
    DeletionPolicy: Retain
    Metadata:
      aws:cdk:path: test-stack/Bucket/Resource
  BucketPolicyE9A3008A:
    Type: AWS::S3::BucketPolicy
    Properties:
      Bucket:
        Ref: Bucket83908E77
      PolicyDocument:
        Statement:
          - Action: s3:GetObject
            Effect: Allow
            Principal:
              AWS: "*"
            Resource:
              Fn::Join:
                - ""
                - - Fn::GetAtt:
                      - Bucket83908E77
                      - Arn
                  - /*
        Version: "2012-10-17"
    Metadata:
      aws:cdk:path: test-stack/Bucket/Policy/Resource
  CDKMetadata:
    Type: AWS::CDK::Metadata
    Properties:
      Analytics: v2:deflate64:H4sIAAAAAAAA/zPSs7DQM1BMLC/WTU7J1s3JTNKrDi5JTM7WAQrFFxvrVTuVJmenlug4p+VBWRAqID8nM7kSIQzh14IE/EtLCkrBOoJSi/NLi5JTa3Xy8lNS9bKK9csMLfQMTYE2ZhVnZuoWleaVZOam6gVBaAB96glZjQAAAA==
    Metadata:
      aws:cdk:path: test-stack/CDKMetadata/Default

CDK CLI Version

2.88.0

Framework Version

No response

Node.js Version

v18.15.0

OS

mac os x

Language

Typescript

Language Version

No response

Other information

No response

pahud commented 1 year ago

aws-rafams commented 1 year ago

Trying to write a very basic construct that deploys a simple website on S3, haven't managed to make it work even with some explicit setting of object ownership and access control. Keep getting Access Denied as the bucket policy does not allow for Put actions. Code used to work before:

export class S3Website extends Construct {
  public readonly websiteBucket: s3.Bucket;

  constructor(scope: Construct, id: string, props: IS3WebsiteProps) {
    super(scope, id);

    // ----------------------------------------------------
    // -                  S3 Bucket                       -
    // ----------------------------------------------------
    this.websiteBucket = new s3.Bucket(this, "Bucket", {
      // ⚙️ bucket config
      versioned: true,
      // 🌐 Website config
      blockPublicAccess: {
        blockPublicAcls: false,
        blockPublicPolicy: false,
        ignorePublicAcls: false,
        restrictPublicBuckets: false,
      },
      publicReadAccess: true,
      objectOwnership: s3.ObjectOwnership.OBJECT_WRITER,
      accessControl: s3.BucketAccessControl.PUBLIC_READ,
      websiteIndexDocument: "index.html",
      websiteErrorDocument: "error.html",
      // What happens when I delete the stack?
      removalPolicy: cdk.RemovalPolicy.DESTROY,
      autoDeleteObjects: true,
    });

    const deployment = new s3deploy.BucketDeployment(this, "DeployWebsite", {
      sources: [props.websiteFilesPath],
      destinationBucket: this.websiteBucket,
      contentLanguage: "en",
      accessControl: s3.BucketAccessControl.PUBLIC_READ,
    });
  }
}

robdmoore commented 1 year ago

I seem to have achieved this for a brand new bucket by using policy rather than ACL:

const bucket = new s3.Bucket(this, id, {
  ...,
  blockPublicAccess: {
    blockPublicAcls: true,
    ignorePublicAcls: true,
    restrictPublicBuckets: false,
    blockPublicPolicy: false,
  }
})

bucket.addToResourcePolicy(
  new PolicyStatement({
    actions: ['s3:GetObject'],
    effect: Effect.ALLOW,
    principals: [new StarPrincipal()],
    resources: [bucket.arnForObjects('*')],
  })
)

aws-rafams commented 1 year ago

I managed to make it work by going to the Amazon S3 Settings page and disabling the account level public access block.

main.ts

#!/usr/bin/env node
import 'source-map-support/register';
import * as cdk from 'aws-cdk-lib';
import * as path from "path";
import { S3Website } from "../lib/s3-website/website";
import { Construct } from "constructs";
import * as s3deploy from "aws-cdk-lib/aws-s3-deployment";

export class MyStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    // Provision S3 Website
    const website = new S3Website(this, "website-deployment", {
      websiteFilesPath: s3deploy.Source.asset(path.join(__dirname, "website-files")),
    });

    new cdk.CfnOutput(this, 'website-url', {
      value: website.websiteBucket.bucketWebsiteUrl
    })
  }
}

const app = new cdk.App();
const stack = new MyStack(app, 'S3WebsiteStack', {});

/lib/s3-webiste/website.ts

import * as cdk from "aws-cdk-lib";
import { Construct } from "constructs";
import * as s3 from "aws-cdk-lib/aws-s3";
import * as s3deploy from "aws-cdk-lib/aws-s3-deployment";
import { readFileSync } from "fs";
import * as path from "path";

export interface IS3WebsiteProps {
  readonly websiteFilesPath: s3deploy.ISource;
}

export class S3Website extends Construct {
  public readonly websiteBucket: s3.Bucket;

  constructor(scope: Construct, id: string, props: IS3WebsiteProps) {
    super(scope, id);

    // ----------------------------------------------------
    // -                  S3 Bucket                       -
    // ----------------------------------------------------
    this.websiteBucket = new s3.Bucket(this, "Bucket", {
      versioned: true,
      blockPublicAccess: {
        blockPublicAcls: false,
        blockPublicPolicy: false,
        ignorePublicAcls: false,
        restrictPublicBuckets: false,
      },
      publicReadAccess: true,
      objectOwnership: s3.ObjectOwnership.BUCKET_OWNER_ENFORCED,
      websiteIndexDocument: "index.html",
      websiteErrorDocument: "error.html",
      removalPolicy: cdk.RemovalPolicy.DESTROY,
      autoDeleteObjects: true,
    });

    const deployment = new s3deploy.BucketDeployment(this, "DeployWebsite", {
      sources: [props.websiteFilesPath],
      destinationBucket: this.websiteBucket,
      contentLanguage: "en",
    });
  }
}

mattbbc commented 11 months ago

I managed to make it work by going to the Amazon S3 Settings page and disabling the account level public access block.

I'd be interested to know if this is good practice or not, and if others that have gotten it to work have also done so by disabling the account-level blocks.

I haven't been able to get the '100% works' solution in the OP to work for me. I'm extra confused as my IAM role in the cli has AdministratorAccess.

dmeehan1968 commented 11 months ago

I managed to make it work by going to the Amazon S3 Settings page and disabling the account level public access block.

I haven't been able to get the '100% works' solution in the OP to work for me. I'm extra confused as my IAM role in the cli has AdministratorAccess.

I just came across this problem, for a bucket already created without public access (nor any specification in the CDK for blocks). On checking my account level blocks, these were all disabled (apparently by default but I've used Amplify to deploy websites before on this account so possible that prompted me to disable the account blocks at some point).

@robdmoore's comment worked for me, but the blockPublicAccess setting they quoted were also required - I initially just tried setting the policy for a specific key prefix and it still got access denied, so added those settings and then it worked.

Note that I didn't try to enable public access for the entire bucket, only for a key prefix, e.g. bucket.arnForObjects('prefix/*').

b-tin commented 8 months ago

I seem to have achieved this with blockPublicAccess and publicReadAccess

"aws-cdk": "^2.125.0"

  const bucket = new s3.Bucket(this.scope, 'FrontendBucket', {
      bucketName: `${this.props.suffixName}-mimic-frontend`,
      removalPolicy: cdk.RemovalPolicy.DESTROY,
      encryption: s3.BucketEncryption.S3_MANAGED,
      websiteIndexDocument: 'index.html',
      objectOwnership: s3.ObjectOwnership.OBJECT_WRITER,
      accessControl: s3.BucketAccessControl.PUBLIC_READ,
      blockPublicAccess: {
        blockPublicAcls: false,
        blockPublicPolicy: false,
        ignorePublicAcls: false,
        restrictPublicBuckets: false,
      },
      publicReadAccess: true,
      cors: [
        {
          allowedOrigins: ['*'],
          allowedMethods: [s3.HttpMethods.GET],
          allowedHeaders: ['*'],
          exposedHeaders: [],
          maxAge: 3000
        }
      ],
      lifecycleRules: [
        {
          abortIncompleteMultipartUploadAfter: cdk.Duration.days(7)
        }
      ]
    });

mattfiocca commented 6 months ago

this is what worked for me:

"aws-cdk": "^2.135.0"

const bucket = new s3.Bucket(this.scope, 'bucket-id', {
  bucketName: 'bucket-name',
  removalPolicy: cdk.RemovalPolicy.DESTROY,
  encryption: s3.BucketEncryption.S3_MANAGED,
  websiteIndexDocument: 'index.html', // <- optional
  objectOwnership: s3.ObjectOwnership.OBJECT_WRITER,
  // accessControl: s3.BucketAccessControl.PUBLIC_READ, <-- NO
  blockPublicAccess: {
    blockPublicAcls: false,
    blockPublicPolicy: false,
    ignorePublicAcls: false,
    restrictPublicBuckets: false,
  },
  // publicReadAccess: true, <-- NO
  cors: [
    {
      allowedOrigins: ['*'],
      allowedMethods: [s3.HttpMethods.GET,s3.HttpMethods.HEAD],
      allowedHeaders: ['*'],
      exposedHeaders: [],
      maxAge: 3000
    }
  ],
  lifecycleRules: [
    {
      abortIncompleteMultipartUploadAfter: cdk.Duration.days(7)
    }
  ]
});

// Use this instead of accessControl and publicReadAccess above
const public_policy = new iam.PolicyStatement({
  actions: ['s3:GetObject'],
  effect: iam.Effect.ALLOW,
  principals: [new iam.AnyPrincipal()],
  resources: [bucket.arnForObjects('*')],
});

bucket.addToResourcePolicy(public_policy);

Davidmec commented 6 months ago

this is what worked for me:

"aws-cdk": "^2.135.0"

const bucket = new s3.Bucket(this.scope, 'bucket-id', {
  bucketName: 'bucket-name',
  removalPolicy: cdk.RemovalPolicy.DESTROY,
  encryption: s3.BucketEncryption.S3_MANAGED,
  websiteIndexDocument: 'index.html', // <- optional
  objectOwnership: s3.ObjectOwnership.OBJECT_WRITER,
  // accessControl: s3.BucketAccessControl.PUBLIC_READ, <-- NO
  blockPublicAccess: {
    blockPublicAcls: false,
    blockPublicPolicy: false,
    ignorePublicAcls: false,
    restrictPublicBuckets: false,
  },
  // publicReadAccess: true, <-- NO
  cors: [
    {
      allowedOrigins: ['*'],
      allowedMethods: [s3.HttpMethods.GET,s3.HttpMethods.HEAD],
      allowedHeaders: ['*'],
      exposedHeaders: [],
      maxAge: 3000
    }
  ],
  lifecycleRules: [
    {
      abortIncompleteMultipartUploadAfter: cdk.Duration.days(7)
    }
  ]
});

// Use this instead of accessControl and publicReadAccess above
const public_policy = new iam.PolicyStatement({
  actions: ['s3:GetObject'],
  effect: iam.Effect.ALLOW,
  principals: [new iam.AnyPrincipal()],
  resources: [bucket.arnForObjects('*')],
});

bucket.addToResourcePolicy(public_policy);

I'm thinking this may be a regression of the cdk, being that you and I both hit this on the same day. Since I had a new setup I also had to make sure that Block Public Access settings for this account was turned off for the account. Something that just gives me code smell vibes.

mattfiocca commented 6 months ago

@Davidmec

I'm thinking this may be a regression of the cdk, being that you and I both hit this on the same day. Since I had a new setup I also had to make sure that Block Public Access settings for this account was turned off for the account. Something that just gives me code smell vibes.

Right after posting my workaround, i stepped my new bucket back to a private one (defaults), and put a cloudfront distro in front of it. The distro in my use case was probably a better move anyway, but its interesting to see all the variation on this issue.

RealLukeMartin commented 6 months ago

I ran into this issue while trying to get a simple frontend app running in an s3 bucket. The band-aid workaround I found was doing an initial run without the publicReadAccess: true set. Like this:

const bucket = new Bucket(this, 'MyBucket', {
  websiteIndexDocument: 'index.html',
  blockPublicAccess: {
    blockPublicPolicy: false,
    blockPublicAcls: false,
    ignorePublicAcls: false,
    restrictPublicBuckets: false,
  },
  // publicReadAccess: true,
});

Then after it successfully completed, I reran the deploy with publicReadAccess: true set and it worked.

const bucket = new Bucket(this, 'MyBucket', {
  websiteIndexDocument: 'index.html',
  blockPublicAccess: {
    blockPublicPolicy: false,
    blockPublicAcls: false,
    ignorePublicAcls: false,
    restrictPublicBuckets: false,
  },
  publicReadAccess: true,
});

linjunpop commented 5 months ago

make it work with

 const createS3Bucket = (scope: Construct, uploaderUserArn: string): cdk.aws_s3.Bucket => {
   const bucket = new s3.Bucket(scope, prefixResource('Bucket'), {
-    accessControl: s3.BucketAccessControl.PRIVATE,
+    blockPublicAccess: {
+      blockPublicAcls: false,
+      blockPublicPolicy: false,
+      ignorePublicAcls: false,
+      restrictPublicBuckets: false,
+    },
     cors: [
       {
         allowedMethods: [s3.HttpMethods.GET, s3.HttpMethods.HEAD],
         allowedOrigins: ['*'],
       },
     ],
+    websiteIndexDocument: 'index.html',
   });

   // allow user to upload files to S3 bucket
   bucket.grantPut(new iam.ArnPrincipal(uploaderUserArn));
+
+  // allow anyone to read
+  bucket.grantRead(new iam.AnyPrincipal());

   return bucket;
 }

aws / aws-cdk