psutil 1.1.0 won't import on Centos 6.4

[GoogleCodeExporter]

The newest version of psutils on pypi doesn't work for us.

>>> import psutil
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "build/bdist.linux-x86_64/egg/psutil/__init__.py", line 74, in <module>
  File "build/bdist.linux-x86_64/egg/psutil/_pslinux.py", line 21, in <module>
  File "build/bdist.linux-x86_64/egg/_psutil_linux.py", line 7, in <module>
  File "build/bdist.linux-x86_64/egg/_psutil_linux.py", line 6, in __bootstrap__
ImportError: 
/root/.python-eggs/psutil-1.1.0-py2.7-linux-x86_64.egg-tmp/_psutil_linux.so: 
undefined symbol: prlimit

>>> import platform
>>> platform.platform()
'Linux-2.6.32-358.el6.x86_64-x86_64-with-centos-6.4-Final'

Original issue reported on code.google.com by arn...@infinidat.com on 1 Oct 2013 at 7:03

[GoogleCodeExporter]

We ran into this as well.  We're using RHEL with a 2.6.32 kernel, which is 
prior to the introduction of the prlimit API.

I've attached a patch that would conditionally include the code that depends on 
prlimit.  The change to psutil/__init__.py is a bit ugly, since it reproduces 
the list of some conditionally defined RLIMIT constants already present in 
psutil/_pslinux.py.  It's also updating locals() which may mess with some code 
inspection, so suggestions are welcome.

For now we've rolled back to the prior release of psutil, but I thought I'd 
take a stab at a patch.

Original comment by m...@matt-good.net on 1 Oct 2013 at 9:12

Attachments:

• prlimit.diff

[GoogleCodeExporter]

Fixed in revision 372bbd8245db. Could you please try latest HG version, run 
tests and report back?

Original comment by g.rodola on 3 Oct 2013 at 7:32

• Changed state: Started
• Added labels: Compatibility, Milestone-1.2.0, OpSys-Linux, Priority-High
• Removed labels: Priority-Medium

[GoogleCodeExporter]

That almost fixed it, but you're using "#ifdef HAS_PRLIMIT" which is always 
true, since on older kernels that's defined, but "false".  I changed it to "#if 
HAS_PRLIMIT" and it's working for us.

diff -r 2192b1ed7a01 psutil/_psutil_linux.c
--- a/psutil/_psutil_linux.c    Thu Oct 03 21:31:17 2013 +0200
+++ b/psutil/_psutil_linux.c    Thu Oct 03 12:49:30 2013 -0700
@@ -99,7 +99,7 @@
 #endif

-#ifdef HAS_PRLIMIT
+#if HAS_PRLIMIT
 /*
  * A wrapper around prlimit(2); sets process resource limits.
  * This can be used for both get and set, in which case extra
@@ -331,7 +331,7 @@
      {"ioprio_set", linux_ioprio_set, METH_VARARGS,
         "Set process I/O priority"},
 #endif
-#ifdef HAS_PRLIMIT
+#if HAS_PRLIMIT
      {"prlimit", linux_prlimit, METH_VARARGS,
         "Get or set process resource limits."},
 #endif
@@ -408,7 +408,7 @@
 #endif

-#ifdef HAS_PRLIMIT
+#if HAS_PRLIMIT
     PyModule_AddIntConstant(module, "RLIM_INFINITY", RLIM_INFINITY);
     PyModule_AddIntConstant(module, "RLIMIT_AS", RLIMIT_AS);
     PyModule_AddIntConstant(module, "RLIMIT_CORE", RLIMIT_CORE);

Original comment by m...@matt-good.net on 3 Oct 2013 at 7:51

[GoogleCodeExporter]

Mmm that doesn't work for me on Ubuntu 12.04 (meaning it thinks prlimit() is 
not available).

Original comment by g.rodola on 3 Oct 2013 at 8:09

[GoogleCodeExporter]

Yeah, I guess this really needs an autoconf-style check to attempt compiling a 
short program that uses prlimit.  Looks like there's a patch proposed for 
Python 3.4 that would add prlimit, using autoconf to define HAVE_PRLIMIT:
http://bugs.python.org/file30685/prlimit2.patch

Unfortunately, I'm not sure what would be recommended to do that with distutils 
extension modules.

Original comment by m...@matt-good.net on 3 Oct 2013 at 9:33

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

I'm not an expert of C related tools but I think we might determine the kernel 
version in setup.py in order to use a macro similar to this:

Extension(
   ...
   define_macros=[('HAVE_PRLIMIT', 1 if get_kernel_ver() > (2, 6, 36) else 0)]
   ...
)

I'm not particularly thrilled at this perspective though because 
get_kernel_ver() has to be rock solid.

Original comment by g.rodola on 3 Oct 2013 at 9:53

[GoogleCodeExporter]

A quick research on Google shows that CentOS is not the only distro who got 
bitten by this:
https://www.google.com/search?q=python+prlimit&oq=python+prlimit&aqs=chrome..69i
57j69i60l2.1625j0&sourceid=chrome&ie=UTF-8#q=psutil+prlimit
I guess we need to fix this quickly. 

Original comment by g.rodola on 3 Oct 2013 at 9:59

[GoogleCodeExporter]

Please check out revision 055e8d480462.
After Googling around I noticed some people who uses #ifdef __NR_prlimit64.
I took a look at latest Linux source code, grepped for "prlimit" (case 
insensitive search) and that looks like the only reasonable definition at hand 
we could use.

man pages also states:

> The  prlimit() system call is available since Linux 2.6.36.  
> Library support is available since glibc 2.13.

I'm not sure how and whether we should check the glibc requirement (my best 
guess is we don't). 
AFAICT latest HG version works fine on my Ubuntu 64-bit.
Please report back how it works for you on CentOS.

Original comment by g.rodola on 3 Oct 2013 at 11:46

[GoogleCodeExporter]

I'm afraid it still looks the same on our CentOS, with revision 055e8d480462.

btw the glibc version on CentOS 6.4 is 2.12. If I understand correctly, you 
won't use this system call if the kernel version is as old...?

Please let me know if I can help from our end.

Original comment by arn...@infinidat.com on 4 Oct 2013 at 8:25

[GoogleCodeExporter]

Still looks the same means you keep getting "undefined symbol: prlimit"?

> btw the glibc version on CentOS 6.4 is 2.12. If I understand 
> correctly, you won't use this system call if the kernel version 
> is as old...?

Currently there are no glib version checks and I don't know if there should be.
In summary I'm not sure what to do. =)
I guess now I'll install CentOS so that at least I can reproduce the problem.
Will report back in a while.

Original comment by g.rodola on 4 Oct 2013 at 7:49

[GoogleCodeExporter]

Ok, tried just now with your same configuration:

[root@centos psutil]# uname -a
Linux centos 2.6.32-358.el6.i686 #1 SMP Thu Feb 21 21:50:49 UTC 2013 i686 i686 
i386 GNU/Linux

...and I'm able to compile and run tests.
prlimit() related functions are not executed as expected.
Can you provide more info on what does not work exactly?

Original comment by g.rodola on 4 Oct 2013 at 8:38

[GoogleCodeExporter]

Still getting "undefined symbol: prlimit" on import.
Attaching full output.

Original comment by arn...@infinidat.com on 4 Oct 2013 at 10:39

Attachments:

• output.txt

[GoogleCodeExporter]

> /root/python/include/python2.7/pyconfig.h:1151:1: warning: "_GNU_SOURCE" 
redefined

I have fixed this warning in a revision which is older than the one you're 
supposed to be using. 
Are you sure this isn't something else?
Please try:

python setup.py clean
python setup.py install
python

...and then in the interpreter:

>>> import os, sys
>>> sys.path.remove(os.getcwd())
>>> import psutil

Original comment by g.rodola on 5 Oct 2013 at 12:30

[GoogleCodeExporter]

I'm using the latest revision (I added 'hg identify' to the output).
That warning doesn't comes up when I use the system Python (2.6.6. I was using 
Python 2.7.5 before). The system python fails to import too - same message...
I'm adding the output again, this time using the system python.

sys.path.remove(os.getcwd()) doesn't work, because sys.path doesn't contain the 
current directory.

Original comment by arn...@infinidat.com on 5 Oct 2013 at 6:32

Attachments:

• output.txt

[GoogleCodeExporter]

If you remove HAVE_PRLIMIT definition, like this, does it still crash?

diff --git a/psutil/_psutil_linux.c b/psutil/_psutil_linux.c
--- a/psutil/_psutil_linux.c
+++ b/psutil/_psutil_linux.c
@@ -27,10 +27,6 @@

  // Linux >= 2.6.13
 #define HAVE_IOPRIO defined(__NR_ioprio_get) && defined(__NR_ioprio_set)
- // Linux >= 2.6.36 kernels, glibc >= 2.13
-#ifdef __NR_prlimit64
-    #define HAVE_PRLIMIT
-#endif

My doubt is that for some reason your python interpreter is using an open 
version of _psutil_linux.so.

Original comment by g.rodola on 5 Oct 2013 at 6:47

[GoogleCodeExporter]

s/open/old

Original comment by g.rodola on 5 Oct 2013 at 6:52

[GoogleCodeExporter]

This resolves the problem, import now succeeds.
I tried to compile a simple program with ifdef, looks like __NR_prlimit64 is 
defined...

I also see this here:

[root@host-ci08 ~]# grep -R prlimit /usr/include/ | grep define
/usr/include/bits/syscall.h:#define SYS_prlimit64 __NR_prlimit64
/usr/include/asm-generic/unistd.h:#define __NR_prlimit64 261
/usr/include/asm/unistd_64.h:#define __NR_prlimit64             302
/usr/include/asm/unistd_32.h:/* #define __NR_prlimit64      340 */

Original comment by arn...@infinidat.com on 5 Oct 2013 at 7:19

[GoogleCodeExporter]

So latest HG version works for you, is that right?

Original comment by g.rodola on 5 Oct 2013 at 7:21

[GoogleCodeExporter]

Negative. The workaround you suggested works; the latest revision on hg does 
not.

Original comment by arn...@infinidat.com on 5 Oct 2013 at 7:32

[GoogleCodeExporter]

I suggested that just to understand whether your python interpreter detected 
the change, it was not a workaround.

OK, I still cannot reproduce the issue and ran out of no ideas. Do you think 
perhaps you can give me SSH access to that box so that I can try it myself?

Original comment by g.rodola on 5 Oct 2013 at 7:48

[GoogleCodeExporter]

I'm getting the same error on Scientific Linux 6.4 (kernel 
v2.6.32-358.18.1.el6.x86_64) with Python 2.7.5 (built locally). Attaching 
output (apologies that some of it is in Swedish but you can probably make out 
the meaning nonetheless). Haven't had time to try the new revision you suggest 
yet but I will probably give it a whirl later today.

Original comment by mariogiov on 7 Oct 2013 at 9:53

Attachments:

• psutil-err_scilinux-6.4.out

[GoogleCodeExporter]

Yes please, use latest HG version (hg clone 
https://g.rodola@code.google.com/p/psutil/) in which I already attempted to 
provide a fix.

Original comment by g.rodola on 7 Oct 2013 at 10:00

[GoogleCodeExporter]

Same import trouble (using commit 1623:b1e6e9da489e). Attaching output for that 
as well. I also tried with Python v2.7.1 but that made no difference. Hmm.

Original comment by mariogiov on 7 Oct 2013 at 10:23

Attachments:

• psutil-err_scilinux-6.4_hg-release.out

[GoogleCodeExporter]

Ok, I should have made it. Please try revision c41b9ce21b04.

Original comment by g.rodola on 7 Oct 2013 at 8:02

[GoogleCodeExporter]

Looks good.
Revision c41b9ce21b04 builds and imports successfully on our CentOS machine.

Original comment by arn...@infinidat.com on 7 Oct 2013 at 9:01

[GoogleCodeExporter]

I released 1.1.1 version including this just now. Closing.

Original comment by g.rodola on 7 Oct 2013 at 11:06

• Changed state: Fixed
• Added labels: Milestone-1.1.1
• Removed labels: Milestone-1.2.0

[GoogleCodeExporter]

verified:

[root@host-ci08 ~]# easy_install -i http://pypi.python.org/simple psutil
Searching for psutil
Reading http://pypi.python.org/simple/psutil/
Reading http://code.google.com/p/psutil/
Best match: psutil 1.1.1
Downloading 
https://pypi.python.org/packages/source/p/psutil/psutil-1.1.1.tar.gz#md5=24430ee
6486be2f1a960d9ce4dc87ad0
Processing psutil-1.1.1.tar.gz
Writing /tmp/easy_install-TA3D6H/psutil-1.1.1/setup.cfg
Running psutil-1.1.1/setup.py -q bdist_egg --dist-dir 
/tmp/easy_install-TA3D6H/psutil-1.1.1/egg-dist-tmp-dSVuFW
zip_safe flag not set; analyzing archive contents...
Adding psutil 1.1.1 to easy-install.pth file

Installed 
/root/python/lib64/python2.7/site-packages/psutil-1.1.1-py2.7-linux-x86_64.egg
Processing dependencies for psutil
Finished processing dependencies for psutil
[root@host-ci08 ~]# ipython
impoer Python 2.7.5 (default, Sep 30 2013, 17:45:26)
Type "copyright", "credits" or "license" for more information.

IPython 1.1.0 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: import psutil

In [2]: psutil.__
psutil.__all__           psutil.__dict__          psutil.__getattribute__  
psutil.__name__          psutil.__reduce__        psutil.__sizeof__
psutil.__builtins__      psutil.__doc__           psutil.__hash__          
psutil.__new__           psutil.__reduce_ex__     psutil.__str__
psutil.__class__         psutil.__file__          psutil.__init__          
psutil.__package__       psutil.__repr__          psutil.__subclasshook__
psutil.__delattr__       psutil.__format__        psutil.__loader__        
psutil.__path__          psutil.__setattr__       psutil.__version__

In [2]: psutil.__version__
Out[2]: '1.1.1'

Original comment by g...@rzn.co.il on 8 Oct 2013 at 12:06

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Sorry but it seems I get the same issue on Oracle Enterprise Linux 6.4
(oraclelinux-release-6Server-4.0.4.x86_64)

Python 2.6.6 and psutil 1.1.1

>>> import psutil
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python2.6/site-packages/psutil-1.1.1-py2.6-linux-x86_64.egg/psutil/__init__.py", line 89, in <module>
    import psutil._pslinux as _psplatform
  File "/usr/lib64/python2.6/site-packages/psutil-1.1.1-py2.6-linux-x86_64.egg/psutil/_pslinux.py", line 21, in <module>
    import _psutil_linux
ImportError: 
/usr/lib64/python2.6/site-packages/psutil-1.1.1-py2.6-linux-x86_64.egg/_psutil_l
inux.so: undefined symbol: prlimit

Any idea ?

Thanks in advance

Original comment by sv24i...@gmail.com on 19 Oct 2013 at 9:40

[GoogleCodeExporter]

Please try revision 5bdae7b71188.

Original comment by g.rodola on 20 Oct 2013 at 11:05

[GoogleCodeExporter]

Hi,

The revision 5bdae7b71188 works !

Thanks

Original comment by sv24i...@gmail.com on 20 Oct 2013 at 2:55