search

matanolabs / matano

Open source security data lake for threat hunting, detection & response, and cybersecurity analytics at petabyte scale on AWS
https://matano.dev
Apache License 2.0
1.44k stars 97 forks source link

s3:PutPublicAccessBlock Access Denied #124

Open ngocsanguit opened 1 year ago

ngocsanguit commented 1 year ago

Thank you team for the great Matano! When I deploy it I got this error

The stack named CDKToolkit failed creation, it may need to be manually deleted from the AWS console: ROLLBACK_COMPLETE: API:
 ›   s3:PutPublicAccessBlock Access Denied

So will the Matano setup create a public s3 bucket/objects? How can we change it only to create in non-public s3?

Samrose-Ahmed commented 1 year ago

Thanks Sang!

So this error is occurring in the CDK bootstrap process, the bucket is not public, it is actually the opposite; its being set as explicitly not public.

The error is occurring due to your organizations policy blocking any usage of the s3:PutPublicAccessBlock action (see the following issue: https://github.com/aws/aws-cdk/issues/5028#issuecomment-553906518).

There is a CDK optioon: --public-access-block-configuration false for the CDK bootstrap command to not explicitly set the public access block configuration.

We also use the block public access inside our own code for internal buckets, e.g. https://github.com/matanolabs/matano/blob/7373558a865f981bea25bd7be23c3a64a36a3fa4/infra/src/DPCommonStack.ts#L32

Amazon S3 announced that all new S3 buckets will have Block Public Access enabled by default, at which point we can safely remove these explicit configurations. However, the change will be rolled out in April 2023 (not yet) (see https://docs.aws.amazon.com/AmazonS3/latest/userguide/create-bucket-faq.html).

Can you possibly change your organization policy to allow usage of the s3:PutPublicAccessBlock action?

ngocsanguit commented 1 year ago

Thank you for the prompt response Samrose. Because PutPublicAccessBlock will allow making an S3 bucket public so it's blocked by our policy. Is there any way that I can remove this explicit setting from the code and run without using PutPublicAccessBlock?

Samrose-Ahmed commented 1 year ago

I understand, we can add a config option in matano.config.yml for this and then make a code change in infrastructure to only add the public access block configuration explicitly if the matano configuration tells us to. I can make this change, should be simple.

Samrose-Ahmed commented 1 year ago

I've created a change #125, once its merged and released, you should be able to unblock yourself by just setting aws.set_block_public_access as false in matano.config.yml e.g.

# matano.config.yml

aws:
  account: "123456789012"
  region: "us-east-1"
  set_block_public_access: false