Open thvasilo opened 1 month ago
Hi @thvasilo , thanks for reaching out.
I tried to repro similar issue with a sample code, which reads files from 2 different folders and does the deployment into target bucket. here is a code snippet and Readme doc for reference
from aws_cdk import (
# Duration,
Stack,
# aws_sqs as sqs,
aws_s3_deployment as s3Deploy,
aws_s3 as s3,
RemovalPolicy,
)
from constructs import Construct
class BucketDeploymentIssueStack(Stack):
def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None:
super().__init__(scope, construct_id, **kwargs)
##source_bucket = s3.Bucket.from_bucket_arn(self, "sourceBucket001","arn:aws:s3:::my-first-bucket-191")
target_bucket = s3.Bucket(self, "targetBucket001",
removal_policy=RemovalPolicy.DESTROY,
auto_delete_objects=True,
block_public_access=s3.BlockPublicAccess.BLOCK_ALL
)
deployment = s3Deploy.BucketDeployment(self, "DeployZippedFolders",
sources= [s3Deploy.Source.asset("./assetSource")],
destination_key_prefix="folder",
destination_bucket=target_bucket,
retain_on_delete=True
)
deployment.add_source(s3Deploy.Source.asset("./assetSource02"))
This is the directory structure -
and this is snippet of copied resources into the bucket -
Hope this is what you are trying to do and this sample code solves your problem. Let me know if it does not work out or your usecase is different.
That worked yes! I create a single deployment and add two sources, instead of creating one deployment per file. I would have expected both cases to work, but this is a good workaround.
My guess is this happens because of step 3 from the readme
The custom resource invokes its associated Lambda function, which downloads the .zip archive, extracts it and issues aws s3 sync --delete against the destination bucket (in this case websiteBucket). If there is more than one source, the sources will be downloaded and merged pre-deployment at this step.
Question: if I had used an existing bucket for this, would running the deployment have deleted all other files on that bucket (assuming I had set it to upload to the bucket root as above). That seems quite dangerous
Yes, that seems correct. Just to verify , I ran this code on existing bucket(created as target bucket above which has the copied zipped files already) -
target_bucket002 = s3.Bucket.from_bucket_arn(self, "targetBucket002", "arn:aws:s3:::bucketdeploymentissuestack-targetbucket001d2a3fb71-xdun4ggp3wrq")
deployment = s3Deploy.BucketDeployment(self, "DeployZippedFolders",
sources= [s3Deploy.Source.asset("./assetSource")],
##destination_key_prefix="folder",
destination_bucket=target_bucket002,
retain_on_delete=True
)
CfnOutput(self, "DeploymentOutput", value=target_bucket002.bucket_arn)
and after re-deploying from another stack, this is whats seen now in the target bucket. looks like other 2 zipped files are deleted with folder.
Let me check with team on this and get back to you
Describe the bug
I have a stack for which I wish to deploy two scripts on S3.
I have two files under
../assets
which I want both deployed at the top level of the createdAssetBucket
. What I'm observing instead is that only one of the files ends up being deployed, and it seems like it's random, sometimes the corresponding toSINGLE_ENTRYPOINT_FILENAME
is deployed, sometimesENTRYPOINT_FILENAME
.Not sure if there's some sort of race condition going on.
Expected Behavior
Both files should be deployed
Current Behavior
Only one of the resource files actually ends up on S3
Reproduction Steps
Here's how I'm trying to accomplish that:
Possible Solution
No response
Additional Information/Context
No response
CDK CLI Version
2.149.0 (build c8e5924)
Framework Version
No response
Node.js Version
v18.18.2
OS
Amazon Linux 2
Language
Python
Language Version
3.9.18
Other information
No response