Provide extended per-process memory stats

[GoogleCodeExporter]

See ml discussion at: 
https://groups.google.com/forum/?fromgroups#!topic/psutil/Yepd8QNTKF0

As of right now psutil is only able to return RSS (resident set size) and VMS 
(virtual memory size) memory stats about a given process.

How the process memory is handled and under what values it is represented 
depends on what platform we're on though, therefore RSS/VMS  alone are not 
enough to represent process memory usage exhaustively.

Under Windows, for example, we have different values we are currently not 
returning:
http://msdn.microsoft.com/en-us/library/windows/desktop/ms684877(v=vs.85).aspx
Same thing for UNIX where we have "shared mem", "library mem", "fault pages" 
and others, depending on the UNIX flavor.

My proposal is to add a new Process.get_ext_memory_info() method returning a 
namedtuple whose attributes depend on what platform we're on. Alternatively 
Process.get_memory_info() can grown an extra "extended" argument defaulting to 
False.

Original issue reported on code.google.com by g.rodola on 23 Apr 2012 at 7:24

[GoogleCodeExporter]

Unfortunately retrieving the private working set can be a bit slow on pre-Vista 
systems, since it involves calling QueryWorkingSet and summing up the private 
pages.

Do you have a list of what attributes you're looking for?

Original comment by wj32...@gmail.com on 23 Apr 2012 at 10:59

[GoogleCodeExporter]

I'm not a Windows guy so I'm not sure what memory stats are more useful than 
others on Windows.
If private pages are the same thing as on Linux then they may be important (see 
issue 96), but for example on Linux I plan to provide "shared" and "private" 
memory stats by iterating over process's shared libraries, as in:

total_pws = 0
for lib in p.get_shared_libs():
    total_pws += lib.private_ws

...instead of in get_memory_info().
I suppose this is not the case on Windows as private working set is not bound 
to process shared libs/modules, am I right?
In this case, yes, it would be good to have the private ws as well.
Are you aware of other memory stats which might be worth including?

Original comment by g.rodola on 24 Apr 2012 at 1:25

[GoogleCodeExporter]

All the stats shown by PE or PH can be useful.

Original comment by wj32...@gmail.com on 25 Apr 2012 at 2:10

[GoogleCodeExporter]

"Unfortunately retrieving the private working set can be a bit slow on 
pre-Vista systems, since it involves calling QueryWorkingSet and summing up the 
private pages."

Windows XP was released in 2001, lets not let it stop us from adding new 
features.

Original comment by sharpbla...@gmail.com on 25 Apr 2012 at 2:55

[GoogleCodeExporter]

Indeed.  On Windows <= XP it would be ok to fallback on using QueryWorkingSet 
or if that's too much work it's also ok to force that specific field to "-1".

Original comment by g.rodola on 25 Apr 2012 at 3:10

[GoogleCodeExporter]

Linux patch in attachment.

Original comment by g.rodola on 8 Jul 2012 at 8:11

Attachments:

• linux-extmem.patch

[GoogleCodeExporter]

Fixed in r1422 (Linux), r1424 (OSX), r1425 (BSD) and r1426 (Windows).
The private working set on Windows is ignored for now.
Perhaps I'll add that later once I figure out a reliable way to check its 
availability at compile time.

Original comment by g.rodola on 10 Jul 2012 at 8:30

• Changed state: FixedInSVN
• Added labels: Milestone-1.0.0

[GoogleCodeExporter]

PrivateUsage on Windows committed as r1427.

Original comment by g.rodola on 10 Jul 2012 at 11:11

[GoogleCodeExporter]

Fixed in version 0.6.0, released just now.

Original comment by g.rodola on 13 Aug 2012 at 4:25

• Changed state: Fixed
• Added labels: Milestone-0.6.0
• Removed labels: Milestone-1.0.0

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Updated csets after the SVN -> Mercurial migration:
r1422 == revision 40f98485c267
r1427 == revision cc26dd332ce7

Original comment by g.rodola on 2 Mar 2013 at 12:07