aws / aws-cdk

The AWS Cloud Development Kit is a framework for defining cloud infrastructure in code
https://aws.amazon.com/cdk
Apache License 2.0
11.6k stars 3.9k forks source link

S3 MFA-Delete and Object Lock support #5247

Open SolDavidCloud opened 4 years ago

SolDavidCloud commented 4 years ago

I don't see how to enforce MFA Delete and Object Locking in an S3 Bucket.

I can't find MFA-Delete in CloudFormation, but ObjectLock is:

Type: AWS::S3::Bucket
Properties: 
  ObjectLockConfiguration: 
    ObjectLockConfiguration
  ObjectLockEnabled: Boolean

Use Case

Define Buckets with MFA-Delete and Object Lock enabled from the start.

Proposed Solution

Add the properties to the S3.Bucket object. Workaround: Doing it manually after creation.

Other

Maybe it already exists, or another workaround exists, and I didn't find it. Thanks.


This is a :rocket: Feature Request

iliapolo commented 4 years ago

@Corsario-Mexico Thanks for reporting this.

Until we push a fix for this, have a look at the Modifying the AWS CloudFormation Resource behind AWS Constructs section in the Escape Hatches Guide.

This allows you to set any CFN supported property on your resource even if it isn't yet supported by the encompassing CDK L2 construct. It is meant as a workaround for exactly these types of scenarios.

iliapolo commented 4 years ago

@Corsario-Mexico Regarding MFA delete, thats a bit more tricky because you are right, there is no official CloudFormation support for that.

This needs to be implemented using custom resources. We will put this on the docket.

btw, if you'd like to contribute this feature that would be awesome. We will of course provide support and guidance all the way. Just let us know :)

ryparker commented 3 years ago

Came across this issue when looking for ways to configure Object Lock. Here's my workaround that uses escape hatches:

import { Bucket, CfnBucket } from '@aws-cdk/aws-s3'

const bucket = new Bucket(stack, 'Bucket', {
  ...
  versioned: true, // Bucket versioning is required when enabling ObjectLock
});

// Get the CloudFormation resource
const cfnBucket = bucket.node.defaultChild as CfnBucket;
// Add the ObjectLockConfiguration prop to the Bucket's CloudFormation output.
cfnBucket.addPropertyOverride('ObjectLockConfiguration.ObjectLockEnabled', 'Enabled');

ObjectLockConfiguration CloudFormation docs

trompx commented 2 years ago

Hey @ryparker,

The problem is that object lock must be set when the bucket is created.

By doing what you suggests, the addPropertyOverride applies on an already existing bucket, thus failing with the following error: Object Lock configuration cannot be enabled on existing buckets (Service: Amazon S3; Status Code: 409; Error Code: InvalidBucketState; Request ID: XXX; S3 Extended Request ID: YYY=; Proxy: null)

Or maybe I am doing something wrong...

EDIT: @iliapolo is there a way to create the CfnBucket (L1 construct) then augments its properties with a Bucket (L2 construct) ? In fact the only way I see to have object lock is to first create it with CfnBucket, but then we cannot use other useful features brought by s3.Bucket L2 construct like autoDeleteObjects.

ryparker commented 2 years ago

If you're importing the bucket using a method such as Bucket.fromBucketName() then you wouldn't be able to change the resource at all. This applies to all resources that are imported into CDK not just Buckets.

Although you can use an imported resource anywhere, you cannot modify the imported resource.

importing resources in CDK


If you're creating a new bucket in your CDK code but already deployed without the object lock override then an attempt to update your CDK code with this override will fail with the mentioned error:

Object Lock configuration cannot be enabled on existing buckets

As far as I know your only option is to delete the bucket and redeploy with the desired property overrides.

trompx commented 2 years ago

Thanks for your input @ryparker But I tried again to cdk deploy after a cdk destroy, making sure the buckets were not existing, with the following setup and am still having the error.

const bucket = new s3.Bucket(this, id, {
  bucketName: bucketName,
  blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL,
  versioned: true,
  publicReadAccess: false,
  autoDeleteObjects: isDev ? true : false,
});
const cfnBucket = bucket.node.defaultChild as s3.CfnBucket;
cfnBucket.addPropertyOverride(
  "ObjectLockConfiguration.ObjectLockEnabled",
  "Enabled"
);
EugRomanchenko commented 2 years ago

Thanks for your input @ryparker But I tried again to cdk deploy after a cdk destroy, making sure the buckets were not existing, with the following setup and am still having the error.

const bucket = new s3.Bucket(this, id, {
  bucketName: bucketName,
  blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL,
  versioned: true,
  publicReadAccess: false,
  autoDeleteObjects: isDev ? true : false,
});
const cfnBucket = bucket.node.defaultChild as s3.CfnBucket;
cfnBucket.addPropertyOverride(
  "ObjectLockConfiguration.ObjectLockEnabled",
  "Enabled"
);

One more parameter overwrite is missing to enable Object Lock (https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_s3.CfnBucketProps.html):


cfnBucket.addPropertyOverride(
  "ObjectLockEnabled",
  true
);
cfnBucket.addPropertyOverride(
  "ObjectLockConfiguration.ObjectLockEnabled",
  "Enabled"
);
zachgoll commented 1 year ago

To summarize the current working solution from all the threads above:

(I've also added a retention period - seems like the CDK stack will get "stuck" without this config)

import { Bucket, CfnBucket } from 'aws-cdk-lib/aws-s3'

const bucket = new Bucket(this, 'ObjLockExampleBucket', {
    versioned: true
})

const bucketResource = bucket.node.defaultChild as CfnBucket
bucketResource.addPropertyOverride('ObjectLockEnabled', true)
bucketResource.addPropertyOverride('ObjectLockConfiguration.ObjectLockEnabled', 'Enabled')

// Can be `Years` or `Days`, not both (integer value)
bucketResource.addPropertyOverride('ObjectLockConfiguration.Rule.DefaultRetention.Years', 5)

// Can be `GOVERNANCE` or `COMPLIANCE` - https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-s3-bucket-defaultretention.html
bucketResource.addPropertyOverride('ObjectLockConfiguration.Rule.DefaultRetention.Mode', 'GOVERNANCE')
laurelmay commented 1 year ago

As of v2.64.0 you can now enable object lock using:

new s3.Bucket(stack, 'Bucket', {
  objectLockEnabled: true,
});

or

new s3.Bucket(stack, 'Bucket', {
  objectLockDefaultRetention: s3.ObjectLockRetention.governance(cdk.Duration.days(5 * 365)),
});

https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_s3-readme.html#object-lock-configuration

It looks like MFA Delete is still not supported by the underlying CloudFormation resources (so no progress there).