aws-samples / cql-replicator

CQLReplicator is a migration tool that helps you to replicate data from Cassandra to AWS Services
Apache License 2.0
15 stars 8 forks source link

Stop using hard-coded S3 bucket name #141

Closed jlewis-spotnana closed 5 months ago

jlewis-spotnana commented 5 months ago

Is your feature request related to a problem? Please describe. I'm using terraform to create the S3 bucket and IAM role for cqlreplicator. Cqlreplicator is storing CQReplicatLor.scala into an S3 bucket named s3://aws-glue-assets-"$AWS_ACCOUNT"-"$AWS_REGION". This bucket name is hard-coded into the cqlreplicator script, frustrating efforts to implement least privilege access control.

Describe the solution you'd like I suggest storing CQReplicatLor.scala into the S3 bucket provided by cli flag --landing-zone.

Describe alternatives you've considered Hard-coding the S3 bucket name into my IAM policy is one possible work-around.

Another work-around is creating two S3 buckets using terraform, one for --landing-zone and another for scala script using the hard-coded S3 bucket name.

Additional context Also, the cqlreplicator --state cleanup feature does not destroy this S3 bucket.

nwheeler81 commented 5 months ago

@jlewis-spotnana please use a flag --main-script-landing during the init phase to store CQReplicator.scala in the landing s3 bucket, e.g. CQLReplicator.scala will be stored in s3://your-landing-bucket/scripts/CQReplicator.scala

jlewis-spotnana commented 4 months ago

It works. Thanks! 🎉