search

rwynn / route81

A bi-directional sync daemon for MongoDB and Kafka
MIT License
9 stars 6 forks source link

Question: Is this supposed to work with Mongo DB 3.2? #6

Open asif-ir opened 4 years ago

asif-ir commented 4 years ago

Change Streams weren't introduced until mongo version 3.6 so is it worth setting this up for syncing a mongo 3.2 DB server?

rwynn commented 4 years ago

This project is really only supporting 3.6 and above via change streams. But you can make a change to this line to flip it to false. This would turn on a legacy mode where change events are read directly from the oplog. This requires the MongoDB user to have access to the collection oplog.rs in database local and that the MongoDB cluster is setup as a replica set (can be standalone/single instance).

asif-ir commented 4 years ago

Yes, noticed those little things. Not sure how stable it really is. There is another project that I found to be more active/popular - GTM. Currently selected that.

The use is that we are migrating a legacy ruby on rails project (which writes to a mongo 3.2 in prod) to newer technologies and during the migration, we want to keep on synchronising the existing mongo writes and raise intermediate events to transform and change the old document to conform to the new schema (Massive changes). And currently GTM seems to be working properly.

But we haven't finalised any tool yet even though GTM is among the favourites. Any suggestions or considerations (about it being able to handle huge load) that you have?