A memory leaks test script is needed

[GoogleCodeExporter]

As far as I know Jay should have wrote one.
Before adding further C code, especially on Windows, a new test script
should be added in the current code base to allow us testing that any
change applied to C extensions does not introduce memory leaks.

Original issue reported on code.google.com by billiej...@gmail.com on 12 Mar 2009 at 12:25

[GoogleCodeExporter]

Original comment by billiej...@gmail.com on 12 Mar 2009 at 12:26

[GoogleCodeExporter]

Actually for memory leak testing I used process_listing.py: 

import psutil
import sys
import os

h = open("process_listing.pid", 'w')
h.write(str(os.getpid()))
h.close()

try: 
    run_count = 0
    while (run_count < 20000):
        run_count = run_count + 1
        sys.stdout.write("\r\r%s" % run_count)
        sys.stdout.flush()

        try:
            cmdline = psutil.Process(0).cmdline
        except Exception, e:
            print "\n"
            print "Exception for PID 0: %s" % e
            continue

        PID = -1
        for i in psutil.get_process_list():
            PID = i.pid
            try:
                cmdline = i.cmdline
            except psutil.NoSuchProcess:
                continue
            except Exception, e:
                print "\n"
                print "Exception for PID %s: %s" % (PID, e)
                continue

except:
    os.remove("process_listing.pid")

I used that to run a loop iterating over as much of the code as possible in the
background by calling str() on a Process object, then used top or ps on UNIX
platforms, or taskmgr on Windows, to monitor the process memory usage. Some 
shifting
of memory usage is normal, as long as it doesn't climb steadily and remains 
somewhere
around a consistent amount it means there are most likely no memory leaks. 

Ideally psutil will support memory statistics in the next release and we can use
psutil itself to monitor for memory leaks in a cross-platform way without 
manually
checking the memory usage. Then we can create a self-contained script called
something like test/memory_usage.py that runs a bunch of iterations (I'd guess 
that a
few thousand is enough to verify a leak) and looks for memory usage climbing 
beyond a
pre-set threshold.

Original comment by jlo...@gmail.com on 12 Mar 2009 at 4:34

• Changed state: Accepted

Attachments:

• process_listing.py

[GoogleCodeExporter]

Putting this as blocked on Issue #38 (per-process memory statistics support) 
for now. 

  http://code.google.com/p/psutil/issues/detail?id=38

Original comment by jlo...@gmail.com on 12 Mar 2009 at 4:38

[GoogleCodeExporter]

Added as r308.
For me it works fine on every platform except Windows where test__str__ and
test_get_pid_list fail (not always but quite often).

Original comment by billiej...@gmail.com on 25 Mar 2009 at 12:18

• Changed state: Started

[GoogleCodeExporter]

Windows wasn't the only platform on which such test was failing since Jay told 
me it
was failing also on OS X every now and then.
We noticed that every time there was a difference in the memory usage it was 
always
of 4096 bytes, no matter how many times the loop was repeated.

r315 modifies the script so that if we don't go over 4096 bytes of difference 
between
step 2 and step 1 the test is considered valid.

As of now I'm not aware of any failure on any platform so I guess we can 
consider
this one fixed.

Original comment by billiej...@gmail.com on 25 Mar 2009 at 1:05

• Changed state: Fixed

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Updated csets after the SVN -> Mercurial migration:
r308 == revision 126d2d3a5674
r315 == revision dcd82bc6d9ff

Original comment by g.rodola on 2 Mar 2013 at 11:47