Leak of Mach ports on OS X

[GoogleCodeExporter]

What steps will reproduce the problem?
I noticed on a machine that uses psutil in a long-lived process, Mach ports 
seem to run low or get completely exhausted. Using DTrace, I was able to verify 
that the psutil-based process was in fact using a large number of Mach ports, 
and it appeared to be leaking them. I examined _psutil_osx.c and found that 
Mach port rights were not being properly managed.

What is the expected output?
Mach ports should not be leaked.

What do you see instead?
Mach ports getting leaked.

What version of psutil are you using? What Python version?
Trunk.

On what operating system? Is it 32bit or 64bit version?
Mac 32 and 64-bit.

Please provide any additional information below.

The justification for these changes are:

- mach_host_self() traps into the kernel via host_self_trap() in 
xnu-2050.7.9/osfmk/kern/ipc_host.c. The subroutine's comments state: "Give the 
caller send rights for his own host port.". This is verified with the 
documentation at xnu-2050.7.9/osfmk/man/mach_host_self.html. The kernel creates 
a copy of the send right, which means that right must be deallocated.

- task_for_pid() returns the Mach task port for a given process. This is leaked 
in two locations and also must be deallocated. The only time a Mach call that 
vends a send right that does not need to be manually deallocated is 
mach_task_self(), which is defined as a macro to a global variable for 
performance in /usr/include/mach/mach_init.h.

The attached patch modifies the OS X C implementation of psutil to properly 
manage port rights. I was not able to figure out a way to run the test suite. 
Please let me know how to do that.

Original issue reported on code.google.com by rsesek@google.com on 27 Sep 2012 at 9:11

Attachments:

• mach_port_leak_fix.patch

[GoogleCodeExporter]

Thanks for the report, we obviously were not aware that the ports had to be 
managed and deallocated after use. I've reviewed and merged the patch in r1543 
to roll this into a future release.

To run the test suite it's just python test/test_psutil.py and 
test/test_memory_leaks.py which just checks memory consumption following 
consecutive runs of psutil methods. It would be good for the future if we can 
develop a test to check for Mach port leaks as well to catch similar issues in 
the test suite. 

Original comment by jlo...@gmail.com on 27 Sep 2012 at 9:21

• Changed state: FixedInSVN

[GoogleCodeExporter]

Thanks for the quick turnaround!

Original comment by rsesek@google.com on 27 Sep 2012 at 10:15

[GoogleCodeExporter]

Out of curiosity, is psutil used internally by Google and for what?

Original comment by g.rodola on 27 Sep 2012 at 10:19

[GoogleCodeExporter]

It is available internally at Google, but I cannot say for what it is used.

Original comment by rsesek@google.com on 1 Oct 2012 at 7:51

[GoogleCodeExporter]

Original comment by g.rodola on 18 Nov 2012 at 6:41

• Added labels: Milestone-0.6.2

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Updated csets after the SVN -> Mercurial migration:
r1543 == revision 9d625f59b839

Original comment by g.rodola on 2 Mar 2013 at 12:13

[GoogleCodeExporter]

Original comment by g.rodola on 8 Apr 2013 at 1:21

• Changed state: FixedInHG

[GoogleCodeExporter]

Original comment by g.rodola on 11 Apr 2013 at 9:17

• Added labels: Milestone-0.7.0
• Removed labels: Milestone-0.6.2

[GoogleCodeExporter]

Original comment by g.rodola on 12 Apr 2013 at 6:21

• Changed state: Fixed