adiom-data / dsync

Database synchronization tool
https://adiom.io
GNU Affero General Public License v3.0
5 stars 1 forks source link

[feature] Add a testconn connector for testing. #71

Closed adiom-mark closed 1 month ago

adiom-mark commented 1 month ago

So the idea here is we want to be able to bootstrap/replay commands so we can test our connectors locally more easily. The testconn works off a directory that contains a bootstrap.json (bulk write) and update.json (like cdc). You can set it up by making it a destination, but normal usage would be as a source. There is an example in the fixture directory.

Implementing the integrity check will make it so that we can use it to directly test an opposing connector, though the initial idea is to bootstrap something like mongo, and then apply updates to mongo and connect two actual connectors. And then to set up some way to easily run tests with a set up like this.

Likely some features to look into here is separating Initial Sync + CDC as that control will be useful for controlling this better as well as supporting the "reverse" feature in a better way.