Resource leak over the lifetime of connection objects

[GoogleCodeExporter]

What steps will reproduce the problem?
1. Obtain cursors via the Connection.cursor() method.  (Defined on 
http://code.google.com/p/ibm-db/source/browse/trunk/IBM_DB/ibm_db/ibm_db_dbi.py#
690)
2. Unless connections are periodically closed, the resulting behaviour will 
look like a memory/resource leak.

What is the expected output? What do you see instead?
I expect the number of "live"/in memory Cursor objects to be be relatively 
close to the number currently open/in scope in my code.

Cursors created with the Connection.cursor() method are added to the 
Connection._cursor_list --- and I can find no code removing items from the list 
outside of Connection.close().

What version of the product are you using? On what operating system?

Please provide any additional information below.
SQLAlchemy appear to use the Connection.cursor() method for creating cursors.
Using DB2 with SQLAlchemy, I observed a large number of Cursor objects that 
were never garbage collected due to them being live/present in the 
Connection._cursor_list lists.  (Indicated by checking with 
http://mg.pov.lt/objgraph/)

I have not checked the details in the source code of SQLAlchemy.

In the concrete instance, the problem could be worked around by periodically 
closing and creating new connections.

Original issue reported on code.google.com by arild.ha...@gmail.com on 11 Sep 2012 at 8:46

[GoogleCodeExporter]

We can't periodically close and create the connections, this will start causing 
another problems.

We can remove the the cursor from _cursor_list in Cursor.close() method, I 
think this should resolve the problem.

Please modify your ibm_db_dbi.py file and add the following code snippet at 
line no 665
>>>
if self.__connection is not None:
   try:
      self.__connection._cursor_list.remove(self)
   except:
      pass
<<<

Please let me know above patch works for you or not.

Original comment by rahul.pr...@in.ibm.com on 11 Sep 2012 at 5:49

• Changed state: Accepted

[GoogleCodeExporter]

Closing the connections is not really a solution, though it is a workaround 
which can be applied by a user of the library, without changing the library 
implementation...

On the trunk, line 665 is inside Connection.close(), not Cursor.close().

Applying the change would introduce a race condition with the way the list is 
iterated over in Connection.close(), lines 662 to 665 --- if a cursor is 
removed from the list while this runs, then that may lead to elements being 
moved to indices already passed in the loop, and the loop code will try to 
access an element past the end of (the current state of) the list...
That part could probably be fixed by rewriting the loop to "for cursor in 
list(self._cursor_list)", i.e. iterate over a copy of the list taken when 
entering the loop --- is there a reason for the current approach of iterating 
over "indices" rather than over the list?

Original comment by arild.ha...@gmail.com on 12 Sep 2012 at 8:41

[GoogleCodeExporter]

sorry, you have to apply the above code snippet after the line no 1090 inside 
the Cusor.close() method.

Original comment by rahul.pr...@in.ibm.com on 12 Sep 2012 at 9:40

[GoogleCodeExporter]

The changes have been included in newly released version of ibm_db-2.0.0. 
Please try new version and let us know if still have same problem.

Original comment by rahul.pr...@in.ibm.com on 24 Sep 2012 at 6:10

[GoogleCodeExporter]

Original comment by rahul.pr...@in.ibm.com on 22 Oct 2012 at 6:05

• Changed state: Fixed