Shopify / ghostferry

The swiss army knife of live data migrations
https://shopify.github.io/ghostferry
MIT License
747 stars 70 forks source link

Investigate removing the cursor and merge it back with the DataIterator #306

Open shuhaowu opened 3 years ago

shuhaowu commented 3 years ago

We used to iterate through the data directly in the DataIterator. At one point, we developed the IterativeVerifier, which caused us to refactor the DataIterator such that the data iteration part became a Cursor, so it can be reused by the IterativeVerifier. With the InlineVerifier superseding the IterativeVerifier, the need for such a Cursor has been eliminated. To simplify the codebase, we should merge the functionality of Cursor back into the DataIterator.

We also need to remove the IterativeVerifier after that. See this outdated PR for reference.