search

thouis / numpy-trac-migration

numpy Trac to github issues migration
2 stars 3 forks source link

Memory leak when an errors are raised (ie. comparison of array and empty list) (Trac #2216) #6005

Open numpy-gitbot opened 11 years ago

numpy-gitbot commented 11 years ago

Original ticket http://projects.scipy.org/numpy/ticket/2216 on 2012-09-17 by trac user Hauke, assigned to unknown.

on 64bit Ubuntu 10.04, virtualenv 1.7.2, Python 2.7.3, Numpy 1.6.2, this

import numpy as np r = np.random.rand(2)
l = [] while True: r == l

causes growing memory usage reproducibly. Could not be reproduced on Python 2.6.6, NumPy 1.4.1.

Also posted in http://stackoverflow.com/questions/12461413/why-does-comparison-of-a-numpy-array-with-a-list-consume-so-much-memory

Regards,

Hauke Holtkamp

numpy-gitbot commented 11 years ago

trac user seberg wrote on 2012-09-18

I have a bit trouble with finding the actual python documentation to clarify this...

It seems like the problem is that throughout numpy when PyErr_SetObject is used, it is assumed that a reference is stolen from the error message, but it is not. So whenever broadcasting, etc. fails the error messages reference counting gets wrong. The fix is rather simple though, just add a Py_DECREF(errmsg); after every single PyErr_SetObject call. This bug effects also current git master.

IE. the same thing occurs with:

while True:
     try: 
         a *= a[:-1]
    except:
       pass

And a few other things (as this is used in quite a few places around numpy). I will probably create a PR on github later today, but if this is it (and it seems like it) its really easy to fix.

numpy-gitbot commented 11 years ago

Title changed from Memory leak in comparison of array with list? to Memory leak when an errors are raised (ie. comparison of array and empty list) by trac user seberg on 2012-09-18