aws-samples / cql-replicator

CQLReplicator is a migration tool that helps you to replicate data from Cassandra to AWS Services
Apache License 2.0
15 stars 8 forks source link

[CQLReplicator on Glue] Handling of replication write error #115

Closed hungfaileung closed 8 months ago

hungfaileung commented 8 months ago

If during the replication, the writes to the target Amazon Keyspaces table failed for some records for some reason, how would cql-replicator handles that?

Example error:

nwheeler81 commented 8 months ago

@hungfaileung the CQLReplicator retires with the exponential backoff strategy (256 retires with the based delay 25ms and 1.1 multiplier), but you should plan ahead the incoming traffic and pre-warm/provision target resources to avoid retires.

hungfaileung commented 8 months ago

@nwheeler81 Is the retry talking about Glue job or individual write to target table?

nwheeler81 commented 8 months ago

@nwheeler81 Is the retry talking about Glue job or individual write to target table? first Individual retries and after Glue job retries (default 1, but you can increase it)

hungfaileung commented 8 months ago

@nwheeler81 Thanks