caosiyang / py-mongo-sync

Oplog-based data sync tool that synchronizes data from a replica set to another deployment, e.g.: standalone, replica set, and sharded cluster.
http://caosiyang.github.io/py-mongo-sync/
MIT License
108 stars 46 forks source link

More records in DST database than SRC database #8

Closed angelluismula closed 7 years ago

angelluismula commented 7 years ago

When I run check.py I get an error like this:


dbs : ['uuid'] src db :
dst db :


RESULT COLL SRC DST

ERR uuid.backlog 195053127 195059804 OK uuid.counters 1 1


RESULT COLL INDEX

OK uuid.backlog _id_1 OK uuid.backlog seq_1 OK uuid.counters _id_1

data: FAILED index: SUCCESS

Mongodb version in both server is 3.0.9 and pymongo==3.5.1

When the sync starts no uuid database exist on DST mongo server.

Any ideas on why this happens?

caosiyang commented 7 years ago

You could watch the log of py-mongo-sync and check if the oplog is lastest or keeps nearly realtime sync through the optime.

When realtime sync, record count of collections ion destination could be extremely close to that on source.

2017-10-19 4:19 GMT+08:00 Ángel Luis Mula Jordán notifications@github.com:

When I run check.py I get an error like this:

dbs : ['uuid'] src db : dst db :

RESULT COLL SRC DST ERR uuid.backlog 195053127 195059804 OK uuid.counters 1 1

RESULT COLL INDEX OK uuid.backlog _id_1 OK uuid.backlog seq_1 OK uuid.counters _id_1

data: FAILED index: SUCCESS

Mongodb version in both server is 3.0.9 and pymongo==3.5.1

When the sync starts no uuid database exist on DST mongo server.

Any ideas on why this happens?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/caosiyang/py-mongo-sync/issues/8, or mute the thread https://github.com/notifications/unsubscribe-auth/ACDicCYd-41_h5FzCm2UBGktsrGTbBZrks5stl08gaJpZM4P-TGI .

-- 曹思阳 15011440694 csy3228@gmail.com

angelluismula commented 7 years ago

seems to be a problem with count() function related with this bug https://jira.mongodb.org/browse/SERVER-19472

After apply db.backlog.validate(true), the counters show the correct values.

Thanks.