Closed GoogleCodeExporter closed 9 years ago
Original comment by g.rodola
on 22 Oct 2010 at 9:15
Is that a 64 bit system?
Original comment by g.rodola
on 22 Oct 2010 at 9:15
Yes, here is output from system profiler:
Model Name: MacBook Pro
Model Identifier: MacBookPro6,2
Processor Name: Intel Core i7
Processor Speed: 2.66 GHz
Number Of Processors: 1
Total Number Of Cores: 2
Original comment by jeffery....@gmail.com
on 22 Oct 2010 at 9:25
Can you show the output of the below command as well?
>>> print psutil.cpu_times(); time.sleep(5); print psutil.cpu_times()
system=3.395108; idle=37.254002; user=4.461708; nice=0.0
system=3.395162; idle=37.254894; user=4.461761; nice=0.0
I'm getting similar results here (not 100% but close, 99.99% which probably
means the issue is the same). Giampaolo, can you check this on any other
platforms you have handy and see if this is OS X only? I'd like to figure out
if this is in the C code or the Python code that handles the CPU calculations.
Thanks
Original comment by jlo...@gmail.com
on 22 Oct 2010 at 9:35
Here is my output:
>>> import psutil, time
>>> print psutil.cpu_times(); time.sleep(5); print psutil.cpu_times()
system=1e-06; idle=4294.967295; user=2.112688; nice=0.0
system=1e-06; idle=4294.967295; user=2.112688; nice=0.0
>>>
Original comment by jeffery....@gmail.com
on 22 Oct 2010 at 9:42
I'll have to look at this some more later but I think the
get_system_cpu_times() is not returning valid results for some reason:
>>> _psutil_osx.get_system_cpu_times(); time.sleep(5);
_psutil_osx.get_system_cpu_times()
(4.4710289999999997, 0.0, 3.4054090000000001, 37.401888)
(4.4710669999999997, 0.0, 3.4054660000000001, 37.402794999999998)
After 5 seconds it's giving almost identical values, and I suspect any
differences in the values are probably from floating point math...
Original comment by jlo...@gmail.com
on 22 Oct 2010 at 9:49
Linux 64bit, 2 cores:
giampaolo@ubuntu:~/svn/psutil$ python -c "import psutil, time; print
psutil.cpu_times(); time.sleep(5); print psutil.cpu_times()"
softirq=33.91; iowait=1355.06; system=3944.62; idle=73988.76; user=5297.51;
irq=0.01; nice=339.7
softirq=33.91; iowait=1355.06; system=3945.14; idle=73997.73; user=5298.01;
irq=0.01; nice=339.72
FreeBSD 7.0 32bit, 1 core:
[root@freebsd ~/psutil]# python -c "import psutil, time; print
psutil.cpu_times(); time.sleep(5); print psutil.cpu_times()"
system=696.9296875; irq=7197.453125; idle=32646.4296875; user=81.4296875;
nice=0.0078125
system=696.984375; irq=7200.296875; idle=32646.4296875; user=83.9765625;
nice=0.0078125
Windows 7 64 (py 32 bit), 1 core:
C:\Users\user\Desktop\svn\psutil>C:\python27\python -c "import psutil, time;
print psutil.cpu_times(); time.sleep(5); print psutil.cpu_times()"
idle=16220.328125; system=1174.375; user=81.71875
idle=16224.90625; system=1174.796875; user=81.734375
Original comment by g.rodola
on 22 Oct 2010 at 10:34
Thanks giampaolo, what about cpu_percent, does it look accurate?
Original comment by jlo...@gmail.com
on 23 Oct 2010 at 1:12
Original comment by jlo...@gmail.com
on 23 Oct 2010 at 1:12
Aside from an initial "100%" which perhaps we could adjust someway results look
ok and match top/taskmgr on BSD/Linux/Windows on both single and double CPUs.
The script:
import time, psutil, threading
def foo():
while 1:
pass
t = threading.Thread(target=foo).start()
while 1:
print psutil.cpu_percent()
time.sleep(1)
In r724 I added a test based on the results you pasted in msg #4 which should
fail on OSX so you can at least use that to test results.
Original comment by g.rodola
on 23 Oct 2010 at 10:52
> In r724 I added a test based on the results you pasted in msg #4 which
> should fail on OSX so you can at least use that to test results.
Thanks... I took a look at this some more last night but I am still working on
it.
I can see the problem (times not updating properly) but I don't know if the
system call is just returning strange data or if something has changed in 10.6
in the way statistics are reported/calculated.
Original comment by jlo...@gmail.com
on 23 Oct 2010 at 3:50
Original comment by jlo...@gmail.com
on 23 Oct 2010 at 3:50
Point is times are updated, but not as much as expected, so I agree with you
that this is likely to be a math problem. Maybe we shouldn't divide per
CLOCKS_PER_SEC as we do on BSD:
- (double)r_load.cpu_ticks[CPU_STATE_USER] / CLOCKS_PER_SEC,
+ (double)r_load.cpu_ticks[CPU_STATE_USER],
Original comment by g.rodola
on 23 Oct 2010 at 5:02
I suspect it's a math problem but I don't know why, or what to use instead of
CLOCKS_PER_SEC... we can't just return it as is because it'd be returning clock
ticks for the values, and our math in the cpu_percent() function depends on
seconds.
In theory, dividing by CLOCKS_PER_SEC is the correct thing to do but it's
evidently not working correctly, so something is out of whack :-/
Original comment by jlo...@gmail.com
on 23 Oct 2010 at 5:17
Should be fixed in r725 now. I've no idea what changed, if anything, but
CLOCKS_PER_SEC (1000000) isn't working correctly and OS X 10.6 seems to be
expecting you to divide CLK_TCK (100) instead. Replacing the division with
CLK_TCK I'm getting results that match Activity Monitor when using the
following test:
python -c 'import time,psutil; time.sleep(5); print psutil.cpu_percent();
time.sleep(5); print psutil.cpu_percent()'
Original comment by jlo...@gmail.com
on 23 Oct 2010 at 6:08
Original comment by g.rodola
on 23 Oct 2010 at 6:45
Using the svn just checked out a few minutes ago, I still receive the
following output:
python -c "import psutil as p, time; print p.__version__; print
p.cpu_percent(); time.sleep(5); print p.cpu_percent()"
0.2.0
100.0
100.0
Is there a minimal example I could run that will expose the low-level behavior?
Original comment by jeffery....@gmail.com
on 25 Oct 2010 at 6:27
> Is there a minimal example I could run that will expose the low-level
behavior?
You could try to run this and paste the output:
import psutil, time
while 1:
print sum(psutil.cpu_times())
time.sleep(1)
On my Linux box (2 CPUs) it prints:
1999.74
2001.74
2003.74
2005.75
2007.76
2009.76
2011.76
2013.76
2015.76
2017.76
2019.76
...
Original comment by g.rodola
on 25 Oct 2010 at 6:38
You can call the cpu times function directly from the _psutil_osx module:
python -c 'import _psutil_osx,time; print _psutil_osx.get_system_cpu_times();
time.sleep(5); print _psutil_osx.get_system_cpu_times()'
(68604.160000000003, 0.0, 47660.730000000003, 490471.07000000001)
(68604.639999999999, 0.0, 47661.330000000002, 490479.98999999999)
Original comment by jlo...@gmail.com
on 25 Oct 2010 at 6:53
Reply to comments 18 and 19:
>>> import psutil, time
>>> while 1:
... print sum(psutil.cpu_times())
... time.sleep(1)
...
46475.37
16110926.24
16110926.24
16110926.24
16110926.24
16110926.24
python -c 'import _psutil_osx,time; print _psutil_osx.get_system_cpu_times(); time.sleep(5); print _psutil_osx.get_system_cpu_times()'
(16064133.439999999, 0.01, 327.67000000000002, 45964.080000000002)
(0.050000000000000003, 0.01, 0.0, 45964.080000000002)
Original comment by jeffery....@gmail.com
on 25 Oct 2010 at 7:31
I'm not sure what to make of the user time in your example above (the first
value) it looks like it rolled over. The idle time is what we use to calculate
CPU utilization and you can see in the example that your idle time is the same
in both iterations. If the idle is the same then you're going to see 100%
utilization.
Can you run this and post the output?
python -c "import _psutil_osx,time
for i in range(0, 25):
time.sleep(2)
print _psutil_osx.get_system_cpu_times()"
I'd like to see what your idle time looks like over a longer period.
Thanks
Original comment by jlo...@gmail.com
on 25 Oct 2010 at 11:28
[deleted comment]
This is a second try to post - apologies if this is a repeat post.
Here is the output from
python -c "import _psutil_osx,time
> for i in range(0, 25):
> time.sleep(2)
> print _psutil_osx.get_system_cpu_times()"
(0.02, 0.01, 0.0, 47930.720000000001)
(0.02, 0.01, 0.0, 47930.720000000001)
(0.02, 0.01, 0.0, 47930.720000000001)
(0.02, 0.01, 0.0, 47930.720000000001)
(0.02, 0.01, 0.0, 47930.720000000001)
(0.02, 0.01, 0.0, 47930.720000000001)
(0.02, 0.01, 0.0, 47930.720000000001)
(0.02, 0.01, 0.0, 47930.720000000001)
(0.02, 0.01, 0.0, 47930.720000000001)
(0.02, 0.01, 0.0, 47930.720000000001)
(0.02, 0.01, 0.0, 47930.720000000001)
(0.02, 0.01, 0.0, 47930.720000000001)
(0.02, 0.01, 0.0, 47930.720000000001)
(0.02, 0.01, 0.0, 47930.720000000001)
(0.02, 0.01, 0.0, 47930.720000000001)
(0.02, 0.01, 0.0, 47930.720000000001)
(0.02, 0.01, 0.0, 47930.720000000001)
(0.02, 0.01, 0.0, 47930.720000000001)
(0.02, 0.01, 0.0, 47930.720000000001)
(0.02, 0.01, 0.0, 47930.720000000001)
(0.02, 0.01, 0.0, 47930.720000000001)
(0.02, 0.01, 0.0, 47930.720000000001)
(0.02, 0.01, 0.0, 47930.720000000001)
(0.02, 0.01, 0.0, 47930.720000000001)
(0.02, 0.01, 0.0, 47930.720000000001)
Original comment by jeffery....@gmail.com
on 26 Oct 2010 at 4:27
Interesting... your CPU times are reporting identical results every time. It's
no wonder CPU util is showing 100% then. I'm kind of at a loss, the cpu times
are being pulled directly from a mach host_statistics() call and the only
calculation/manipulation done is to divide it by CLK_TCK. It looks to me like
the host_statics call is returning the same values every time.
I'll put together a small C app you can try it with and see what happens if we
print the raw idle value from host_statistics()
Original comment by jlo...@gmail.com
on 26 Oct 2010 at 4:48
Ok r739 now fixes some warnings and adds error checking around
host_statistics() in get_system_cpu_times() - while making a test C app I
noticed that if host_statistics() is failing the CPU times struct can still
return values. I suspect what may have been happening in your case is the call
to host_statistics is failing and thus not updating the idle time on subsequent
runs. You can try the newest SVN version and see if that makes any difference.
I've also attached a cpu_idle.c file you can compile with just "gcc cpu_idle.c"
if you've got the commandline xcode/gcc tools installed. Then just run ./a.out
and you should see something like this, updated every two seconds:
[user@host ]$ ./a.out
idle=55495234, idle/CLK_TCK=554952.340000
idle=55495585, idle/CLK_TCK=554955.850000
idle=55495925, idle/CLK_TCK=554959.250000
idle=55496269, idle/CLK_TCK=554962.690000
idle=55496604, idle/CLK_TCK=554966.040000
idle=55496927, idle/CLK_TCK=554969.270000
To stop just use CTRL C
Thanks,
-Jay
Original comment by jlo...@gmail.com
on 26 Oct 2010 at 5:19
Attachments:
Here's some output from cpu_idle.c:
gmail:Downloads kline$ ./cpu_idle
idle=175914686, idle/CLK_TCK=1759146.860000
idle=175915425, idle/CLK_TCK=1759154.250000
idle=175916170, idle/CLK_TCK=1759161.700000
idle=175916911, idle/CLK_TCK=1759169.110000
idle=175917680, idle/CLK_TCK=1759176.800000
Original comment by jeffery....@gmail.com
on 26 Oct 2010 at 5:56
Okay - with the latest svn, the output from the code of Comment 21 follows. It
is different from before:
python -c "import _psutil_osx,timefor i in range(0, 25):
time.sleep(2)
print _psutil_osx.get_system_cpu_times()"
(49156.760000000002, 0.0, 37154.800000000003, 1760902.1299999999)
(49156.980000000003, 0.0, 37155.07, 1760909.6000000001)
(49157.220000000001, 0.0, 37155.349999999999, 1760917.0900000001)
(49157.489999999998, 0.0, 37155.669999999998, 1760924.53)
(49157.720000000001, 0.0, 37155.940000000002, 1760931.97)
(49158.050000000003, 0.0, 37156.349999999999, 1760939.3)
(49158.279999999999, 0.0, 37156.769999999997, 1760946.5600000001)
Output from cpu_percent() is unchanged from before:
python -c "import psutil; print psutil.cpu_percent()"
100.0
Original comment by jeffery....@gmail.com
on 26 Oct 2010 at 6:07
> Output from cpu_percent() is unchanged from before:
> python -c "import psutil; print psutil.cpu_percent()"
> 100.0
Bear in mind you have to have the time.sleep() in there, since the CPU util
calculation is performed by comparing the amount of idle time reported against
the amount of clock time reported. The cpu idle values look to be incrementing
nicely now... can you try
python -c 'import time,psutil; time.sleep(5); print psutil.cpu_percent();
time.sleep(5); print psutil.cpu_percent()'
And see if you get something accurate as compared to Activity Monitor?
thanks!
Original comment by jlo...@gmail.com
on 26 Oct 2010 at 6:29
Yes, that's it! Thank you!
Since cpu_percent is often paired with a 'time.sleep' call, I wonder if it
would be simpler to have cpu_percent accept a "sleep" argument with default
behavior of, say, 2s.
Original comment by jeffery....@gmail.com
on 26 Oct 2010 at 6:44
Possibly that would make sense.. at the moment what we do is as soon as you
import the module we start the clock so to speak, so as long as you wait a
tenth of a second or more before calling the cpu_percent() code it should be
able to calculate the CPU percentage. The longer you wait between calls the
more data we have to work with so it'll be more accurate.
I'll discuss the sleep param with Giampaolo and see if we want to go that
route, in the meantime just be sure to give it a little time between calls,
even a tenth of a second or two should be enough.
Original comment by jlo...@gmail.com
on 26 Oct 2010 at 7:08
Original comment by jlo...@gmail.com
on 27 Oct 2010 at 3:39
Original comment by g.rodola
on 13 Nov 2010 at 3:14
[deleted comment]
Updated csets after the SVN -> Mercurial migration:
r261 == revision 7bda2b4aa692
r724 == revision f621399fbd84
r725 == revision 8be414279d25
r739 == revision 2ba734d2027c
Original comment by g.rodola
on 2 Mar 2013 at 11:55
Original issue reported on code.google.com by
jeffery....@gmail.com
on 22 Oct 2010 at 9:08