yugabyte / yb-voyager

Data migration Engine for YugabyteDB database
36 stars 10 forks source link

Bug Fix: remove from conflict cache when ignoring event #1712

Closed makalaaneesh closed 1 week ago

makalaaneesh commented 2 weeks ago

In live migration phase, for every event, we

  1. add to conflictDetectionCache if applicable
  2. process event
  3. remove from conflictDetectionCache if applicable

The issue: As part of step2, we can either add the event to batch and apply it on target DB OR we can ignore the event (because we had already applied it, for example). Step3 is only being executed if we add the event to the batch. In case the event is ignored, it remains in the conflict detection cache forever, causing import-data to wait hang and wait forever. This usually can happen when you stop-start import-data (resumption)

The fix: Run step3 even in cases where we ignore the event.