search

scylladb / scylla-cdc-java

Apache License 2.0
24 stars 15 forks source link

Fixes to consuming CDC log #26

Closed avelanarius closed 3 years ago

avelanarius commented 3 years ago

This PR is dependent on #20 (therefore this many commits - only the last three are related to this PR).

This PR fixes two issues related to consuming CDC log:

  1. consume() method of Transport called from multiple threads, instead from a single thread. consume() method could be called from multiple threads (in some cases Driver thread). This PR fixes this by splitting a single TaskAction, which read using CQL and called consume(), into two TaskActions: one that reads using CQL and second that calls consume().

  2. Reader not resuming from lastConsumedChangeId Previously, Reader would not resume from lastConsumedChangeId - it would read the entire window, potentially causing duplicates. This PR adds skipping of changes up to lastConsumedChangeId.