search

PFZheng / psutil

Automatically exported from code.google.com/p/psutil
Other
0 stars 0 forks source link

Please support GNU/kFreeBSD #202

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago 
Hi,
in debian we have 2 flavors of GNU/kFreeBSD (amd64 and i386) and currently 
psutil is unable to detect those OSes since the check is too restrictive.

With the attached patch I'm able to build psutil and start the test suite, but 
I currently have 2 hangs:

- test_psutil.py - it hangs at "test_cmdline (__main__.TestCase) ..."
- test/test_memory_leaks.py - it hangs at "test_get_pid_list 
(__main__.TestModuleFunctionsLeaks) ..."

I'll see if I'll be able to debug it further in the coming days.

Regards,
Sandro

Original issue reported on code.google.com by matrixhasu on 28 Aug 2011 at 11:22

GoogleCodeExporter commented 9 years ago 
now really attaching the patch

Original comment by matrixhasu on 28 Aug 2011 at 11:23

Attachments:

GoogleCodeExporter commented 9 years ago 
For today I've come up to the attached diff: it seems like several tests are 
hanging due to get_pid_list() . Currently it's implemented using a sysctl, but 
maybe the parameter passed are not the right ones for kFreeBSD. I've asked on 
the support IRC channel and they suggest to use a different approach: 
kvm_getprocs . the the kvm library interface is provided by most (of not all?) 
BSD descendants (f.e. FreeBSD, NetBSD, OpenBSD, DragonFlyBSD, Darwin, 
GNU/kFreeBSD too of course :) and Solaris too) - what do you think about it?

After skipping the hanging tests, there are still a lot of them failing (I'll 
try to nail them down in the coming days, if you don't beat me first ;)):

======================================================================
ERROR: test_connection_fromfd (__main__.TestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_psutil.py", line 147, in inner
    return fun(self, *args, **kwargs)
  File "test/test_psutil.py", line 1009, in test_connection_fromfd
    for conn in p.get_connections():
  File "/home/morph/psutil-read-only/build/lib.gnukfreebsd-8.2-1-amd64-x86_64-2.6/psutil/__init__.py", line 402, in get_connections
    return self._platform_impl.get_connections()
  File "/home/morph/psutil-read-only/build/lib.gnukfreebsd-8.2-1-amd64-x86_64-2.6/psutil/_psbsd.py", line 198, in get_connections
    return lsof.get_process_connections()
  File "/home/morph/psutil-read-only/build/lib.gnukfreebsd-8.2-1-amd64-x86_64-2.6/psutil/_psposix.py", line 184, in get_process_connections
    stdout = self.runcmd(cmd)
  File "/home/morph/psutil-read-only/build/lib.gnukfreebsd-8.2-1-amd64-x86_64-2.6/psutil/_psposix.py", line 253, in runcmd
    raise NotImplementedError(msg)
NotImplementedError: this functionnality requires lsof command line utility to 
be installed on the system

======================================================================
ERROR: test_get_open_files (__main__.TestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_psutil.py", line 911, in test_get_open_files
    files = p.get_open_files()
  File "/home/morph/psutil-read-only/build/lib.gnukfreebsd-8.2-1-amd64-x86_64-2.6/psutil/__init__.py", line 394, in get_open_files
    return self._platform_impl.get_open_files()
  File "/home/morph/psutil-read-only/build/lib.gnukfreebsd-8.2-1-amd64-x86_64-2.6/psutil/_psbsd.py", line 191, in get_open_files
    return lsof.get_process_open_files()
  File "/home/morph/psutil-read-only/build/lib.gnukfreebsd-8.2-1-amd64-x86_64-2.6/psutil/_psposix.py", line 142, in get_process_open_files
    stdout = self.runcmd(cmd)
  File "/home/morph/psutil-read-only/build/lib.gnukfreebsd-8.2-1-amd64-x86_64-2.6/psutil/_psposix.py", line 253, in runcmd
    raise NotImplementedError(msg)
NotImplementedError: this functionnality requires lsof command line utility to 
be installed on the system

======================================================================
ERROR: test_get_open_files2 (__main__.TestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_psutil.py", line 950, in test_get_open_files2
    for path, fd in p.get_open_files():
  File "/home/morph/psutil-read-only/build/lib.gnukfreebsd-8.2-1-amd64-x86_64-2.6/psutil/__init__.py", line 394, in get_open_files
    return self._platform_impl.get_open_files()
  File "/home/morph/psutil-read-only/build/lib.gnukfreebsd-8.2-1-amd64-x86_64-2.6/psutil/_psbsd.py", line 191, in get_open_files
    return lsof.get_process_open_files()
  File "/home/morph/psutil-read-only/build/lib.gnukfreebsd-8.2-1-amd64-x86_64-2.6/psutil/_psposix.py", line 142, in get_process_open_files
    stdout = self.runcmd(cmd)
  File "/home/morph/psutil-read-only/build/lib.gnukfreebsd-8.2-1-amd64-x86_64-2.6/psutil/_psposix.py", line 253, in runcmd
    raise NotImplementedError(msg)
NotImplementedError: this functionnality requires lsof command line utility to 
be installed on the system

======================================================================
ERROR: test_nice (__main__.TestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_psutil.py", line 851, in test_nice
    p.nice = first_nice
  File "/home/morph/psutil-read-only/build/lib.gnukfreebsd-8.2-1-amd64-x86_64-2.6/psutil/__init__.py", line 210, in nice
    return self._platform_impl.set_process_nice(value)
  File "/home/morph/psutil-read-only/build/lib.gnukfreebsd-8.2-1-amd64-x86_64-2.6/psutil/_psbsd.py", line 90, in wrapper
    raise AccessDenied(self.pid, self._process_name)
AccessDenied: (pid=3099)

======================================================================
ERROR: test_zombie_process (__main__.TestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_psutil.py", line 1233, in test_zombie_process
    meth()
  File "/home/morph/psutil-read-only/build/lib.gnukfreebsd-8.2-1-amd64-x86_64-2.6/psutil/__init__.py", line 402, in get_connections
    return self._platform_impl.get_connections()
  File "/home/morph/psutil-read-only/build/lib.gnukfreebsd-8.2-1-amd64-x86_64-2.6/psutil/_psbsd.py", line 198, in get_connections
    return lsof.get_process_connections()
  File "/home/morph/psutil-read-only/build/lib.gnukfreebsd-8.2-1-amd64-x86_64-2.6/psutil/_psposix.py", line 184, in get_process_connections
    stdout = self.runcmd(cmd)
  File "/home/morph/psutil-read-only/build/lib.gnukfreebsd-8.2-1-amd64-x86_64-2.6/psutil/_psposix.py", line 253, in runcmd
    raise NotImplementedError(msg)
NotImplementedError: this functionnality requires lsof command line utility to 
be installed on the system

======================================================================
ERROR: test_get_pids (_posix.PosixSpecificTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/morph/psutil-read-only/test/_posix.py", line 113, in test_get_pids
    if BSD: return
NameError: global name 'BSD' is not defined

======================================================================
ERROR: test_avail_virtmem (_bsd.BSDSpecificTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/morph/psutil-read-only/test/_bsd.py", line 110, in test_avail_virtmem
    _, sysctl_avail_virtmem = parse_sysctl_vmtotal(result)
  File "/home/morph/psutil-read-only/test/_bsd.py", line 33, in parse_sysctl_vmtotal
    total, active = mobj.groups()
AttributeError: 'NoneType' object has no attribute 'groups'

======================================================================
ERROR: test_disks (_bsd.BSDSpecificTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/morph/psutil-read-only/test/_bsd.py", line 146, in test_disks
    usage = psutil.disk_usage(part.mountpoint)
  File "/home/morph/psutil-read-only/build/lib.gnukfreebsd-8.2-1-amd64-x86_64-2.6/psutil/__init__.py", line 667, in disk_usage
    return _psplatform.get_disk_usage(path)
  File "/home/morph/psutil-read-only/build/lib.gnukfreebsd-8.2-1-amd64-x86_64-2.6/psutil/_psposix.py", line 93, in get_disk_usage
    st = os.statvfs(path)
OSError: [Errno 2] No such file or directory: '/srv/chroot/sid/home'

======================================================================
ERROR: test_total_virtmem (_bsd.BSDSpecificTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/morph/psutil-read-only/test/_bsd.py", line 89, in test_total_virtmem
    sysctl_total_virtmem, _ = parse_sysctl_vmtotal(result)
  File "/home/morph/psutil-read-only/test/_bsd.py", line 33, in parse_sysctl_vmtotal
    total, active = mobj.groups()
AttributeError: 'NoneType' object has no attribute 'groups'

======================================================================
FAIL: test_disk_partitions (__main__.TestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_psutil.py", line 412, in test_disk_partitions
    self.assertTrue(os.path.isdir(disk.mountpoint))
AssertionError

======================================================================
FAIL: test_get_num_threads (__main__.TestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_psutil.py", line 693, in test_get_num_threads
    self.assertEqual(step2, step1 + 1)
AssertionError: 1 != 2

======================================================================
FAIL: test_get_threads (__main__.TestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_psutil.py", line 708, in test_get_threads
    self.assertEqual(len(step2), len(step1) + 1)
AssertionError: 1 != 2

======================================================================
FAIL: test_sys_cpu_times2 (__main__.TestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_psutil.py", line 331, in test_sys_cpu_times2
    self.fail("difference %s" % difference)
AssertionError: difference 1.29999999956e-05

======================================================================
FAIL: test_sys_per_cpu_times2 (__main__.TestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_psutil.py", line 356, in test_sys_per_cpu_times2
    self.fail()
AssertionError

======================================================================
FAIL: test_BOOT_TIME (_bsd.BSDSpecificTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/morph/psutil-read-only/test/_bsd.py", line 58, in test_BOOT_TIME
    self.assertEqual(btime, psutil.BOOT_TIME)
AssertionError: 1314001403 != 1314001408.0

======================================================================
FAIL: test_process_create_time (_bsd.BSDSpecificTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/morph/psutil-read-only/test/_bsd.py", line 127, in test_process_create_time
    self.assertEqual(start_ps, start_psutil)
AssertionError: 'Mon Aug 29 21:01:47 2011' != 'Mon Aug 29 21:01:48 2011'

----------------------------------------------------------------------
Ran 82 tests in 2.652s

FAILED (failures=7, errors=9)
/usr/lib/python2.6/atexit.py:24: RuntimeWarning: 
TestCase.test_get_connections_all was skipped
  func(*targs, **kargs)
/usr/lib/python2.6/atexit.py:24: RuntimeWarning: TestCase.test_get_connections 
was skipped
  func(*targs, **kargs)
/usr/lib/python2.6/atexit.py:24: RuntimeWarning: Couldn't run limited user 
tests (super-user privileges are required)
  func(*targs, **kargs)

Additionally, after the test suite completes, the terminal control is not 
returned, and not even Ctrl-C works (Ctrl-Z + kill %x does the trick).

Original comment by matrixhasu on 29 Aug 2011 at 9:24

Attachments:

GoogleCodeExporter commented 9 years ago 
I'll try to reimplement get_pid_list() by using this kvm_getprocs() to fix 
get_pid_list() related failures and let you know once done.

As for test_get_num_threads and test_get_threads failures you might try to add 
a time.sleep(.1) call here:
http://code.google.com/p/psutil/source/browse/trunk/test/test_psutil.py#698
...and here:
http://code.google.com/p/psutil/source/browse/trunk/test/test_psutil.py#712

Open files/connections failures require lsof to be installed (hopefully I'll 
manage to re-implement that part in C and finally get rid of the stinky lsof 
python parser).

Other BSD-specific tests such as test_avail_virtmem and others just need to be 
adapted a bit or simply be skipped if there's a reliable way to detect we're on 
GNU/kFreeBSD (sys.platform?).

As for test_sys_cpu_times* I honestly have no clue. I have the code in front of 
me and I don't understand why it fails (it shouldn't).

Generally speaking, these failures are not that terrible and suggest that 
adding support for these arcane system I've never heard of might worth the 
effort. =)

>     st = os.statvfs(path)
> OSError: [Errno 2] No such file or directory: '/srv/chroot/sid/home'

This is interesting. Maybe it worths opening a report against bugs.python.org.

Original comment by g.rodola on 29 Aug 2011 at 9:53

GoogleCodeExporter commented 9 years ago 
awesome! If you need some help or so I can handle you think ml:
debian-bsd@lists.debian.org (it's actually for Debian porting, but
they can help you on the bsd side of psutil).

even with the sleep I got:

FAIL: test_get_num_threads (__main__.TestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_psutil.py", line 711, in test_get_num_threads
    self.assertEqual(step2, step1 + 1)
AssertionError: 1 != 2

======================================================================
FAIL: test_get_threads (__main__.TestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_psutil.py", line 727, in test_get_threads
    self.assertEqual(len(step2), len(step1) + 1)
AssertionError: 1 != 2

maybe you can check the existance of lsof and skip the test if not there?

$ python -c "import sys; print sys.platform"
gnukfreebsd8

I'll give it a look on the machine, maybe I'll be lucky ;)

yeah, kfreebsd is very interesting since it's the only open source
solution with native zfs support (on linux is available under fuse,
which is kinda... slow...). Oh did I tell you we also have hurd (and
psutil will need work for that too, but we'll get to that later)?

It may just be a problem on with the chroot I was using or so - need
to investigate futher.

Original comment by matrixhasu on 30 Aug 2011 at 10:15

GoogleCodeExporter commented 9 years ago

Original comment by jlo...@gmail.com on 16 Sep 2011 at 6:54

GoogleCodeExporter commented 9 years ago 
python -c "import sys; print sys.platform"
openbsd5

    Complete output from command python setup.py egg_info:
    platform openbsd5 is not supported

Original comment by m...@akamit.com on 29 Jan 2013 at 2:52

GoogleCodeExporter commented 9 years ago 
Yes, OpenBSD is currently not supported.

Original comment by g.rodola on 29 Jan 2013 at 2:53

GoogleCodeExporter commented 9 years ago 
Mozilla imported psutil as part of the buildsystem & mach, so other real bsds 
(and debian/kfreebsd) ought to be supported. What should be the way to go, 
implement a backend for each of them, or try to stash everything in the 
existing bsd backend, with if/switches wrt the family ?

Some work was done a while ago for an older version (0.4.1) in 
https://github.com/jasperla/openbsd-wip/blob/master/sysutils/py-psutil/patches/p
atch-psutil__psbsd_py, i'll try to reapply that on top of master.

Original comment by landry.b...@gmail.com on 2 Jun 2013 at 8:12

GoogleCodeExporter commented 9 years ago 
Glad to hear Mozilla adopted psutil! 

In order to add support for a new OS/architecture one should usually file an 
issue on the bug tracker so that we can keep track of the progress and provide 
a patch.

Alternatively I can give you commit privileges against the Mercurial repository 
and create a separate branch where you can develop and once everything is ready 
merge it back into trunk.

This issue is for GNU/kFreeBSD so if you're willkng to work on adding OpenBSD 
support please file a separate issue and we'll keep discussing there.

Thanks,

- Giampaolo

Original comment by g.rodola on 2 Jun 2013 at 8:24

GoogleCodeExporter commented 9 years ago 
psutil has been migrated from Google Code to Github (see: 
http://grodola.blogspot.com/2014/05/goodbye-google-code-im-moving-to-github.html
).
Please do NOT reply here but use this instead:
https://github.com/giampaolo/psutil/issues/202

Original comment by g.rodola on 26 May 2014 at 3:07

GoogleCodeExporter commented 9 years ago 
Has there been any progress on this front?

Original comment by g.rodola on 2 Jan 2015 at 4:27