disk_usage raises UnicodeEncodeError when called with unicode string

[GoogleCodeExporter]

>>> from __future__ import unicode_literals
>>> import os
>>> from psutil import disk_usage
>>> name = "ƒőő"
>>> os.mkdir(name)
>>> disk_usage(name)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/kparmar/.virtualenvs/tmp/lib/python2.7/site-packages/psutil/__init__.py", line 1196, in disk_usage
    return _psplatform.get_disk_usage(path)
  File "/Users/kparmar/.virtualenvs/tmp/lib/python2.7/site-packages/psutil/_psposix.py", line 99, in get_disk_usage
    st = os.statvfs(path)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-2: 
ordinal not in range(128)

What version of psutil are you using? What Python version?
psutil v1.0.1
python v2.7.3

On what operating system? Is it 32bit or 64bit version?
OS X 10.8.4
Win 7 Enterprise 64-bit

Original issue reported on code.google.com by kunalpar...@gmail.com on 9 Aug 2013 at 9:17

[GoogleCodeExporter]

Tested this on Python 3.3 and everything works fine while on Python 2.7 I can 
work around the error by using name.encode('utf8').
It looks like the difference in behavior between mkdir and statvfs is caused by 
how they treat the paths given as input:

- mkdir: 
http://hg.python.org/cpython/file/0152152b09d0/Modules/posixmodule.c#l2651
- statvfs: 
http://hg.python.org/cpython/file/0152152b09d0/Modules/posixmodule.c#l7449

This looks like a Python bug to me (statvfs should use PyArg_ParseTuple(args, 
"et")).
I will file an issue on the Python bug tracker.
As for what concerns psutil I don't think we should take any action.

Original comment by g.rodola on 9 Aug 2013 at 10:04

[GoogleCodeExporter]

I tried using name.encode('utf8') on Windows, but that did not work -

$ cat test.py
# coding: utf-8

from __future__ import unicode_literals
import os
from psutil import disk_usage
name = "ƒőő"
os.mkdir(name)
disk_usage(name.encode('utf8'))

$ python test.py
Traceback (most recent call last):
  File "test.py", line 8, in <module>
    disk_usage(name.encode('utf-8'))
  File "c:\users\sync\sync\vendor\psutil\psutil\__init__.py", line 1020, in disk
_usage
    return _psplatform.get_disk_usage(path)
  File "c:\users\sync\sync\vendor\psutil\psutil\_psmswindows.py", line 98, in ge
t_disk_usage
    raise OSError(errno.ENOENT, "No such file or directory: '%s'" % path)
OSError: [Errno 2] No such file or directory: 'ƒőő'

Original comment by kunalpar...@gmail.com on 9 Aug 2013 at 10:39

[GoogleCodeExporter]

Ah OK! Well, I think we can at least fix that on Windows.

Original comment by g.rodola on 9 Aug 2013 at 11:43

• Added labels: Priority-Medium
• Removed labels: Priority-High

[GoogleCodeExporter]

Windows fixed in revision 9d233da8d725.
os.statvfs() bug is now tracked at: http://bugs.python.org/issue18695.
At this point I think psutil should expose its own fixed version of statvfs() 
on Python 2 and keep using the one provided by the os module on Python 3.

Original comment by g.rodola on 9 Aug 2013 at 12:11

[GoogleCodeExporter]

I added a pure python workaround for POSIX as of revision c8451ddcf746.
Re-exposing statvfs() from scratch was too much work.
I consider this fixed.

Original comment by g.rodola on 9 Aug 2013 at 1:26

• Changed state: FixedInHG
• Added labels: Milestone-1.1.0

[GoogleCodeExporter]

Shameless Plug: I open sourced a project that we have used quite successfully 
to test our application for Unicode characters. You may find it useful to test 
all psutil APIs. Its available at https://pypi.python.org/pypi/rotunicode.

Original comment by kunalpar...@gmail.com on 22 Aug 2013 at 7:34

[GoogleCodeExporter]

Original comment by g.rodola on 28 Sep 2013 at 10:06

• Changed state: Fixed

[GoogleCodeExporter]

Closing out as fixed. Release 1.1.0 is now available for download.

Original comment by g.rodola on 28 Sep 2013 at 5:32