Currently, the YBPartition class object returns a partition ID by the method getId() which depends whether the table is in colocated:
colocated: tableId.tabletId
non colocated: tabletId
So all the offsets, keys, OpIds, etc. are stored against this ID.
Now since the service had special handling for snapshot for colocated tables, we treat every YBPartition as a colocated object in the snapshot phase - what that means is that we can get the snapshot GetChangesResponse for each colocated table individually. While we treat every YBPartition in streaming phase as non-colocated since we do not have a way to get the changes for each tablet separately.
This creates a problem at the management level that we have to take care of properly creating the YBPartition object otherwise we'll end up returning a wrong ID if we use YBPartition#getId method.
Solution
This PR removes the colocated property from the YBPartition class and introduces the following changes
YBPartition#getID will now return full partition ID every time, this makes the ID a consistent value across snapshot and streaming phase.
If a table is colocated and we're in the streaming phase, we will update the offset values for all the tables belonging to the colocated tablet upon every GetChanges call.
The tabletSafeTime map is now moved to YugabyteDBOffsetContext so that it can be handled consistently for all the change event source classes.
Test plan
Ran existing tests for colocated tables as well as non colocated tables to ensure this PR breaks nothing.
Problem
Currently, the
YBPartition
class object returns a partition ID by the methodgetId()
which depends whether the table is in colocated:tableId.tabletId
tabletId
So all the offsets, keys, OpIds, etc. are stored against this ID.
Now since the service had special handling for snapshot for colocated tables, we treat every
YBPartition
as a colocated object in the snapshot phase - what that means is that we can get the snapshotGetChangesResponse
for each colocated table individually. While we treat everyYBPartition
in streaming phase as non-colocated since we do not have a way to get the changes for each tablet separately.This creates a problem at the management level that we have to take care of properly creating the
YBPartition
object otherwise we'll end up returning a wrong ID if we useYBPartition#getId
method.Solution
This PR removes the colocated property from the
YBPartition
class and introduces the following changesYBPartition#getID
will now return full partition ID every time, this makes the ID a consistent value across snapshot and streaming phase.GetChanges
call.tabletSafeTime
map is now moved toYugabyteDBOffsetContext
so that it can be handled consistently for all the change event source classes.Test plan
Ran existing tests for colocated tables as well as non colocated tables to ensure this PR breaks nothing.