search

aws-samples / cql-replicator

CQLReplicator is a migration tool that helps you to replicate data from Cassandra to AWS Services
Apache License 2.0
16 stars 8 forks source link

[CQLReplicator on Glue] cassandra to AWS keyspaces replication - ran out of available IP address #113

Closed frozensky closed 9 months ago

frozensky commented 9 months ago

Describe the bug Ran out of IP address on the subnet

To Reproduce Steps to reproduce the behavior:

  1. run cqlreplicator with 40 tiles and each tiles running 56 workers, I assume each worker needs an IP address. I have a /21 subnet which only have around 2000 IP available.

  2. See error JobName:CQLReplicator and JobRunId:jr_fdeebc7dd067bea37ec946a9a83c2fe8d0b3da0e9b48f945e8561f7c0503eafc failed to execute with exception Number of IP addresses on subnet is 0: subnet-XXXXXXXXX (Service: AWSGlueJobExecutor; Status Code: 400; Error Code: InvalidInputException; Request ID: 410f9143-7427-423e-bd9c-3510d9136c2d; Proxy: null)

Screenshot 2024-02-08 at 12 42 22 AM

frozensky commented 9 months ago

testing create 2nd connection on different subnet and attached to the job

nwheeler81 commented 9 months ago

@frozensky I can add a preflight check against subnet-XXXXXXXXX to verify the number of available IPs.

frozensky commented 9 months ago

adding 2nd connection to the job works