search

aws-solutions / aws-waf-security-automations

This solution automatically deploys a single web access control list (web ACL) with a set of AWS WAF rules designed to filter common web-based attacks.
https://aws.amazon.com/solutions/aws-waf-security-automations
Apache License 2.0
852 stars 367 forks source link

Step 5, Upload deployment assets issue (NoSuchBucket) #201

Closed BeatKO closed 2 years ago

BeatKO commented 3 years ago

Describe the bug Hi I'm declaring successfully environment variables in step 4, but in step "5 , Upload deployment assets" the first copy works fine but when i run the second CP command i get this error:

upload failed: deployment/regional-s3-assets/ip_retention_handler.zip to s3://test-test-test-template-nv-us-east-1/test-test-test/1/ip_retention_handler.zip An error occurred (NoSuchBucket) when calling the PutObject operation: The specified bucket does not exist

But, if i change the variable "$DIST_OUTPUT_BUCKET-$AWS_REGION" with the bucket name, the command run successfully.

aijunpeng commented 3 years ago

Thanks for reporting the issue. Per step 3 in readme, the distribution bucket is a regional bucket that stores lambda source code. Therefore the bucket name should have the region name (where you intend to deploy the solution) as its suffix. Step 5 meanly provides a sample command that uploads assets to the $DIST_OUTPUT_BUCKET-$AWS_REGION bucket that you created previously. If all the steps are followed, the command should succeed.

BeatKO commented 2 years ago

I did everything according to the instructions, please notice that when i don't user the variable "$DIST_OUTPUT_BUCKET-$AWS_REGION"

The command run successfully

aijunpeng commented 2 years ago

It seems the error message shows that the s3 path has a prefix "s3://test-test-test-template-nv-us-east-1/test-test-test/1" instead of bucket name. $DIST_OUTPUT_BUCKET-$AWS_REGION should be only the bucket name.

calvinchiulele1 commented 2 years ago

It worked for me by setting the following values:

It seems like the DIST_OUTPUT_BUCKET variable must have the region name twice for it to work because when I set the region name once in the variable (like this export DIST_OUTPUT_BUCKET = web-application-firewall-skssnsjs-af-south-1), the CF stack returned an error specifying that the bucket does not exist, which leads to assuming that the CF template requires that the region name must be set twice in the bucket name, and it kept failing until I put the region name twice in the variable just for testing if it'd work and it did.

After setting those variables, the CF stack created the resources successfully.

Doesn't the documentation lack explanation on how to set the dist output bucket name properly to avoid having this issue?

aijunpeng commented 2 years ago

Thanks for the comment. We will revisit the relevant instructions in future releases.

calvinchiulele1 commented 2 years ago

A better resolution for the issue is the following:

# This assumes all of the OS-level configuration has been completed and git repo has already been cloned 
# 
# This script should be run from the repo's deployment directory 
# cd deployment 
# ./build-s3-dist.sh source-bucket-base-name trademarked-solution-name version-code 
# 
# Paramenters: 
#  - template-bucket: Name for the S3 bucket location where the templates are found
#  - source-bucket-base-name: Name for the S3 bucket location where the Lambda source 
#    code is deployed. The template will append '-[region_name]' to this bucket name.
#  - trademarked-solution-name: name of the solution for consistency 
#  - version-code: version of the package 
#
#    For example: ./build-s3-dist.sh template-bucket source-bucket-base-name my-solution v3.0
#    The template will then expect the source code to be located in the solutions-[region_name] bucket

For finding the solution to the problem, I had to deep dive into the source code in the aws-waf-security-automations-main/deployment/build-s3-dist.sh file to read it thoroughly and understand how it sets the env variables, and according to the following line source-bucket-base-name: Name for the S3 bucket location where the Lambda source code is deployed. The template will append '-[region_name]' to this bucket name., the source-bucket-base-name env variable mustn't have the region name appended to it, as the template will append it by itself. Whenever I appended the region name to that variable, the CF stack failed to create indicating that the S3 bucket doesn't exist, therefore, that variable must only contain the name of the global bucket, which is a prefix for it. In the line The template will then expect the source code to be located in the solutions-[region_name] bucket, it "implies" that the source-bucket-base-name parameter must match the trademarked-solution-name parameter, which is the same as SOLUTION_NAME env variable, hence the DIST_OUTPUT_BUCKET and SOLUTION_NAME env variables values must match.

I created the S3 buckets with the following names:

and I set the env variables for the solution as follows (where AWS_REGION varied for the CloudFront endpoint):

I uploaded the deployment assets with the same commands described in the README.md, and it ran successfully.

Therefore, for avoiding the issue, I set the same values for TEMPLATE_OUTPUT_BUCKET, DIST_OUTPUT_BUCKET, and SOLUTION_NAME env variables, whether it's for the ALB endpoint or the CloudFront endpoint, and the CF stack executes successfully.