scylladb / scylla-cdc-java

Apache License 2.0
24 stars 15 forks source link

Always use prepared statements for replication #13

Open kbr- opened 3 years ago

kbr- commented 3 years ago

https://github.com/scylladb/scylla-cdc-java/blob/24265007f5ab1fdfccefeea4fa6516ffc55d8666/scylla-cdc-replicator/src/main/java/com/scylladb/cdc/replicator/ReplicatorConsumer.java#L71-L83

We can always use prepared statements. Sometimes we may need to use a batch statement, but still. Example prepared statement that can handle any non-frozen map update:

begin unlogged batch
update ks.t set v = v + :added, v = v - :removed where pk = :pk and ck = :ck;
update ks.t set v = :replaced where pk = :pk and ck = :ck;
apply batch

bind the markers as needed: :added from the CDC v column, :removed from cdc$deleted_elements_v, :replaced from cdc$deleted_v (bind null if it got deleted, leave unset otherwise).

kbr- commented 3 years ago

Also until https://github.com/scylladb/scylla/issues/7825 is fixed, always bind :removed in the above example (can bind to empty set if no elements got removed)

kbr- commented 3 years ago

In general, I think that a batch of 2 can solve every case:

for cdc$operation = 1 (update):

for cdc$operation = 2 (insert):