In disconnect: RuntimeError: dictionary changed size during iteration

tonal commented 5 years ago

TxMongo version: 18.2.0 PyMongo version: 3.7.2 Python 3.6.7 MongoDB version: 4.0.3

My code:

  def spider_closed(self, spider):
    if self.connect:
      connect, self.connect = self.connect, None
      disconnect = getattr(connect, 'disconnect', None)
      if disconnect:
        return disconnect() ### <<< 102 line

Stack trace:

RuntimeError: dictionary changed size during iteration
  File "/opt/pyenv/versions/3.6.7/envs/amon/lib/python3.6/site-packages/twisted/internet/defer.py", line 151, in maybeDeferred
  File "/opt/pyenv/versions/3.6.7/envs/amon/lib/python3.6/site-packages/pydispatch/robustapply.py", line 55, in robustApply
  File "/opt/amon/amon/amon/mongo_ext.py", line 102, in spider_closed
  File "/opt/pyenv/versions/3.6.7/envs/amon/lib/python3.6/site-packages/txmongo/connection.py", line 342, in disconnect

Error caught on signal handler: <bound method MongoExtension.spider_closed of <amon.mongo_ext.MongoExtension object at 0x7f154d3b83c8>>

May be Line 342 in txmongo/connection.py:

for pinger in self.__pingers.values():

change to

for pinger in list(self.__pingers.values()):

IlyaSkriblovsky commented 5 years ago

Good catch, thanks! Does the problem appear when you create a txmongo connection and then immediately disconnect without sending any queries?

Can you make a pull request for this?

tonal commented 5 years ago

@IlyaSkriblovsky Not. Query exists. 2 create_index (all exists) and 1 find. ~10sec. between connect and disconnect.

twisted / txmongo

In disconnect: RuntimeError: dictionary changed size during iteration #243