OperationFailure: database error: getMore: cursor didn't exist on server, possible restart or timeout?

[thekorn]

I read about this tool recently and tried to sync two mongodb sharded clusters with mongo-connector today. This somehow does not work, and crashes with a traceback.

$ python mongo_connector.py -m XXX.edelight.net:27017 -t mongodb://localhost -d ./doc_managers/mongo_doc_manager.py
2012-08-16 08:48:04,332 - INFO - Beginning Mongo Connector
2012-08-16 08:48:05,377 - INFO - MongoConnector: Empty oplog progress file.
2012-08-16 08:48:05,440 - INFO - OplogManager: Initializing oplog thread
2012-08-16 08:48:05,444 - INFO - MongoConnector: Starting connection thread Connection([u's1.edelight.net:27017', u's2.edelight.net:27017', u's3.ede
light.net:27017'])
2012-08-16 08:48:05,472 - INFO - OplogManager: Initializing oplog thread
2012-08-16 08:48:05,475 - INFO - MongoConnector: Starting connection thread Connection([u's4.edelight.net:27017', u's5.edelight.net:27017', u'6.ede
light.net:27017'])
Exception in thread Thread-2:
Traceback (most recent call last):
  File "/usr/lib/python2.6/threading.py", line 532, in __bootstrap_inner
    self.run()
  File "/home/mkorn/mongo-connector/oplog_manager.py", line 101, in run
    cursor = self.init_cursor()
  File "/home/mkorn/mongo-connector/oplog_manager.py", line 322, in init_cursor
    timestamp = self.dump_collection()
  File "/home/mkorn/mongo-connector/oplog_manager.py", line 273, in dump_collection

    for doc in cursor:
  File "/usr/local/lib/python2.6/dist-packages/pymongo-2.2-py2.6-linux-x86_64.egg/pymongo/cursor.py", line 747, in next
    if len(self.__data) or self._refresh():
  File "/usr/local/lib/python2.6/dist-packages/pymongo-2.2-py2.6-linux-x86_64.egg/pymongo/cursor.py", line 711, in _refresh
    limit, self.__id))
  File "/usr/local/lib/python2.6/dist-packages/pymongo-2.2-py2.6-linux-x86_64.egg/pymongo/cursor.py", line 657, in __send_message
    self.__tz_aware)
  File "/usr/local/lib/python2.6/dist-packages/pymongo-2.2-py2.6-linux-x86_64.egg/pymongo/helpers.py", line 102, in _unpack_response
    error_object["$err"])
OperationFailure: database error: getMore: cursor didn't exist on server, possible restart or timeout?

Exception in thread Thread-3:
Traceback (most recent call last):
  File "/usr/lib/python2.6/threading.py", line 532, in __bootstrap_inner
    self.run()
  File "/home/mkorn/mongo-connector/oplog_manager.py", line 101, in run
    cursor = self.init_cursor()
  File "/home/mkorn/mongo-connector/oplog_manager.py", line 322, in init_cursor
    timestamp = self.dump_collection()
  File "/home/mkorn/mongo-connector/oplog_manager.py", line 273, in dump_collection
    for doc in cursor:
  File "/usr/local/lib/python2.6/dist-packages/pymongo-2.2-py2.6-linux-x86_64.egg/pymongo/cursor.py", line 747, in next
    if len(self.__data) or self._refresh():
  File "/usr/local/lib/python2.6/dist-packages/pymongo-2.2-py2.6-linux-x86_64.egg/pymongo/cursor.py", line 711, in _refresh

    limit, self.__id))
  File "/usr/local/lib/python2.6/dist-packages/pymongo-2.2-py2.6-linux-x86_64.egg/pymongo/cursor.py", line 657, in __send_message
    self.__tz_aware)
  File "/usr/local/lib/python2.6/dist-packages/pymongo-2.2-py2.6-linux-x86_64.egg/pymongo/helpers.py", line 102, in _unpack_response
    error_object["$err"])
OperationFailure: database error: getMore: cursor didn't exist on server, possible restart or timeout?

At this point it hangs forever, and the target system does not get any data.

XXX.edelight.net points to a mongos in the old cluster. On this server I see incoming connections in the mongodb logs.

If you need further information I'm happy to provide them, this mongo-manager seems like a promising solution for various scenarios to me.

Markus

[stedile]

Hi Markus,

Try pip installing the latest version of Mongo Connector and see if the problem persists (it was updated today). Also make sure you have the latest version of pymongo (2.2) installed. If you're still getting the error consistently, could you send the the logs for mongos? Which version of python are you using?

[thekorn]

Hi Leonardo, great, your recent changes seem to fixed the problem, version 1.0.0 is working so far.

Thanks again Markus