Closed mojodna closed 6 years ago
I'm still a bit fuzzy on how Offsets are handled. For the (pending) AugmentedDiffSource, a current sequence of 3094146 results in 3094145.json being fetched.
Offset
AugmentedDiffSource
3094146
3094145.json
Similarly, commit() is called with values that I don't expect.
commit()
Offset handling in ReplicationStreamMicroBatchReader is likely to blame: https://github.com/azavea/osmesa/blob/bf8565602ff13256281cfc611fb9ced4b1a3d397/src/common/src/main/scala/osmesa/common/streaming/ReplicationStreamMicroBatchReader.scala#L60-L99
ReplicationStreamMicroBatchReader
I'm still a bit fuzzy on how
Offsets are handled. For the (pending)AugmentedDiffSource, a current sequence of3094146results in3094145.jsonbeing fetched.Similarly,
commit()is called with values that I don't expect.Offsethandling inReplicationStreamMicroBatchReaderis likely to blame: https://github.com/azavea/osmesa/blob/bf8565602ff13256281cfc611fb9ced4b1a3d397/src/common/src/main/scala/osmesa/common/streaming/ReplicationStreamMicroBatchReader.scala#L60-L99