scylladb / scylla-cdc-java

Apache License 2.0
24 stars 15 forks source link

Non-frozen list inserts are replicated incorrectly #11

Closed kbr- closed 3 years ago

kbr- commented 3 years ago

As far as I understand, the replicator replicates these operations using INSERT, which causes Scylla to generate the list's timeuuid keys internally - in particular, the operation is not idempotent.

Non-frozen list modifications must always be replicated using UPDATE ... SET l[SCYLLA_TIMEUUID_LIST_INDEX(...)] = ..., so that the key is provided by the client, which makes the operation idempotent and ensures that the order of elements in the replicated list is the same as in the original list.

To replicate an INSERT operation, we can do something like this:

begin unlogged batch
insert into ks.t (pk, ck) values (0, 0);
update ks.t set v[scylla_timeuuid_list_index(...)] = ... where pk = 0 and ck = 0;
apply batch

so we use a separate insert to create the row marker, but update to create the list elements.

kbr- commented 3 years ago

github tricked me into opening the issue twice