search

caosiyang / py-mongo-sync

Oplog-based data sync tool that synchronizes data from a replica set to another deployment, e.g.: standalone, replica set, and sharded cluster.
http://caosiyang.github.io/py-mongo-sync/
MIT License
108 stars 46 forks source link

Syncing from standalone server without auth to replica set with auth returns replset error #15

Closed claflico closed 6 years ago

claflico commented 6 years ago

Trying to sync data from a single mongo server without authentication to a replica set with authentication and am seeing the following:

conf.toml

[src]
hosts = "10.50.51.95:17017"

[dst]
type = "mongo"

[dst.mongo]
hosts = "10.50.51.90:47017"
authdb = "admin"
username = "root"
password = "ROOT_PASSWORD"

[sync]
[[sync.dbs]]
db = "test-db"

Output:

/opt/py-mongo-sync # python sync.py -f conf.toml
2018-06-29 18:39:40,583 INFO ================================================
2018-06-29 18:39:40,583 INFO src hostportstr :  10.50.51.95:17017
2018-06-29 18:39:40,583 INFO src authdb      :  admin
2018-06-29 18:39:40,583 INFO src username    :
2018-06-29 18:39:40,583 INFO src password    :
2018-06-29 18:39:40,610 INFO src db version  :  3.4.15
2018-06-29 18:39:40,610 INFO dst hostportstr :  10.50.51.90:47017
2018-06-29 18:39:40,610 INFO dst authdb      :  admin
2018-06-29 18:39:40,610 INFO dst username    :
2018-06-29 18:39:40,610 INFO dst password    :
2018-06-29 18:39:40,642 INFO dst db version  :  3.4.14
2018-06-29 18:39:40,643 INFO databases       :  iam-uat
2018-06-29 18:39:40,643 INFO collections     :  iam-uat.*
2018-06-29 18:39:40,643 INFO db mapping      :
2018-06-29 18:39:40,643 INFO fileds          :
2018-06-29 18:39:40,643 INFO start optime    :  None
2018-06-29 18:39:40,643 INFO optime logfile  :
2018-06-29 18:39:40,643 INFO log filepath    :
2018-06-29 18:39:40,643 INFO asyncio         :  True
2018-06-29 18:39:40,643 INFO pymongo version :  3.7.0
2018-06-29 18:39:40,643 INFO ================================================
Traceback (most recent call last):
  File "sync.py", line 29, in <module>
    syncer.run()
  File "/opt/py-mongo-sync/mongosync/synchronizer.py", line 67, in run
    self._sync()
  File "/opt/py-mongo-sync/mongosync/synchronizer.py", line 87, in _sync
    oplog_start = get_optime(self._src.client())
  File "/opt/py-mongo-sync/mongosync/mongo_utils.py", line 111, in get_optime
    rs_status = mc['admin'].command({'replSetGetStatus': 1})
  File "/usr/local/lib/python2.7/site-packages/pymongo/database.py", line 614, in command
    codec_options, session=session, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/pymongo/database.py", line 514, in _command
    client=self.__client)
  File "/usr/local/lib/python2.7/site-packages/pymongo/pool.py", line 579, in command
    unacknowledged=unacknowledged)
  File "/usr/local/lib/python2.7/site-packages/pymongo/network.py", line 150, in command
    parse_write_concern_error=parse_write_concern_error)
  File "/usr/local/lib/python2.7/site-packages/pymongo/helpers.py", line 155, in _check_command_response
    raise OperationFailure(msg % errmsg, code, response)
pymongo.errors.OperationFailure: not running with --replSet

Also note how in the output the src authdb shows admin even though I didn't specify one in the toml file and the dst username & dst password are empty even though one is specified on the conf file.

caosiyang commented 6 years ago

The source should be a replica set, becasuse py-mongo-sync will read the oplogs and replay on the destination. Standalone mongo server has no 'oplog.rs' in 'local', so it cannot work.