Linux / Python3 implementation speedup

[GoogleCodeExporter]

I noticed a substantial difference between Python 2.7 and 3.4 in terms of speed 
on Linux.
Apparently all psutil functions are considerably slower on Python 3.4, 
sometimes by a factor of 2x!
Some examples:

giampaolo@UX32VD:~/svn/psutil$ python2.7 -m timeit -s "import psutil" 
"psutil.cpu_times(True)"
10000 loops, best of 3: 51 usec per loop
giampaolo@UX32VD:~/svn/psutil$ python3.4 -m timeit -s "import psutil" 
"psutil.cpu_times(True)"
10000 loops, best of 3: 87.9 usec per loop

giampaolo@UX32VD:~/svn/psutil$ python2.7 -m timeit -s "import psutil" 
"psutil.virtual_memory()"
10000 loops, best of 3: 29.5 usec per loop
giampaolo@UX32VD:~/svn/psutil$ python3.4 -m timeit -s "import psutil" 
"psutil.virtual_memory()"
10000 loops, best of 3: 60 usec per loop

giampaolo@UX32VD:~/svn/psutil$ python2.7 -m timeit -s "import psutil" 
"psutil.swap_memory()"
10000 loops, best of 3: 52.9 usec per loop
giampaolo@UX32VD:~/svn/psutil$ python3.4 -m timeit -s "import psutil" 
"psutil.swap_memory()"
10000 loops, best of 3: 81.4 usec per loop

This is true for both system and process functions.
The culprit is the way files are opened.
By default "open(file, 'r')" on Python 2.7 opens the file in binary mode 
whereas on Python 3.X the file is opened in text mode.
That means that every time we read a line Python will convert it in 
text/unicode by using system default encoding.
Since Linux implementation of psutil relies almost entirely on reading files in 
/proc the slowdown affects the entire API.

Original issue reported on code.google.com by g.rodola on 15 Feb 2014 at 3:00

[GoogleCodeExporter]

Fixed in revision f8c40e7ecbab.
Some benchmarks:

===============================================================================
function                              1.2.1       2.0.0           unit
===============================================================================
psutil.cpu_percent                      101        44.5           msec per loop
psutil.cpu_times                       59.2          41           usec per loop
psutil.cpu_times_percent                101        76.5           msec per loop
psutil.disk_io_counters                 213         228           usec per loop
psutil.disk_partitions                  103        80.1           usec per loop
psutil.disk_usage                      6.02         6.4           usec per loop
psutil.get_boot_time                     53        39.3           usec per loop
psutil.get_pid_list                     236         199           usec per loop
psutil.get_users                       50.8        52.3           usec per loop
psutil.net_io_counters                 97.1         100           usec per loop
psutil.pid_exists                      1.13        1.26           usec per loop
psutil.process_iter                    11.4        9.45           msec per loop
psutil.swap_memory                     81.6        57.7           usec per loop
psutil.virtual_memory                  57.7        44.3           usec per loop
===============================================================================

Process class methods benefit of similar speedups.
Closing this out.

Original comment by g.rodola on 22 Feb 2014 at 1:34

• Changed state: FixedInHG

[GoogleCodeExporter]

Closing out as fixed as 2.0.0 version is finally out.

Original comment by g.rodola on 10 Mar 2014 at 11:36

• Changed state: Fixed
• Added labels: Milestone-2.0.0