search

aws-samples / cql-replicator

CQLReplicator is a migration tool that helps you to replicate data from Cassandra to AWS Services
Apache License 2.0
15 stars 8 forks source link

Bugs within shell script #133

Closed mati999q closed 6 months ago

mati999q commented 6 months ago

Bug 1

Describe the bug After the latest update, when initialising a new CQL Replicator env, the glue connection is made with the naming convention cql-replicator-{uuid}$DEFAULT_ENV

The glue job which is created then appends the $DEFAULT_ENV to the end of this name, and tries to use it as a connection. This results in an error

JobName:CQLReplicatorENV and JobRunId:xxx failed to execute with exception Could not find connection for the given criteria Failed to get catalog connections given names: cql-replicator-uuidENVENV (Service: AWSGlueJobExecutor; Status Code: 400; Error Code: InvalidInputException; Request ID: xxx Proxy: null)

To Reproduce Steps to reproduce the behavior:

  1. Initialise any new CQL replicator environment where DEFAULT_ENV is not empty
  2. Run the discovery job for this environment (or simply see the Glue job and Glue connection are not linked)
  3. See error in the AWS Glue console

Expected behavior A clear and concise description of what you expected to happen.

CQL Replicator should be able to provide a working environment when first created.

Screenshots

image

Additional context Difficult when explaining how to use the tool, to add one more thing to explain and change / I managed to fix by removing an extra $DEFAULT_ENV when creating glue job

Bug 2

Describe the bug When creating bucket in s3, region flag is not present by default resulting in an errors

[2024-04-03T12:40:46+01:00] Creating a new S3 bucket: s3://cql-replicator-xxx-us-west-2showingerror
[2024-04-03T12:40:47+01:00] ERROR: not able to create a S3 bucket: s3://cql-replicator-xxx-us-west-2showingerror

To Reproduce Steps to reproduce the behavior:

  1. Initialise any new CQL replicator environment where the s3 bucket needs to be created
  2. Notice error in the terminal window

Expected behavior A clear and concise description of what you expected to happen.

CQL Replicator should be able to provide a working environment when first created.

Additional context Difficult when explaining how to use the tool, to add one more thing to explain and change / I managed to fix by adding in a --region $AWS_REGION flag inside the aws s3 mb command, though wish it was already there when upgrading to new version

nwheeler81 commented 6 months ago

Hi @mati999q thanks for reporting the issues, I'm able to reproduce the bug1, but the bug2 is quite specific, you might missing AWS_REGION environmental variable, but I can supply all aws s3 calls with AWS REGION.