UnicodeDecodeError in _read_line()

[pfrenssen]

Some entries in my MPD library seem to cause a UnicodeDecodeError. I'm using Python3.

Script:

#!/usr/bin/python

import mpd 

client = mpd.MPDClient()
client.connect("localhost", 6600)
print(client.mpd_version)

client.iterate = True
allinfo = client.listall()

for item in allinfo:
  print(item)

client.close
client.disconnect()

Output:

....
{'file': 'Reggae - Ragga - Ska/Asian Dub Foundation/2000 - Asian Dub Foundation - Community Music/10 - Asian Dub Foundation - The Judgement.mp3'}
{'file': 'Reggae - Ragga - Ska/Asian Dub Foundation/2000 - Asian Dub Foundation - Community Music/11 - Asian Dub Foundation - Truth Hides.mp3'}
Traceback (most recent call last):
  File "./mpd-test.py", line 12, in <module>
    for item in allinfo:
  File "/usr/lib/python3.2/site-packages/mpd.py", line 304, in _iterator_wrapper
    for item in iterator:
  File "/usr/lib/python3.2/site-packages/mpd.py", line 278, in _read_objects
    for key, value in self._read_pairs():
  File "/usr/lib/python3.2/site-packages/mpd.py", line 260, in _read_pairs
    pair = self._read_pair(separator)
  File "/usr/lib/python3.2/site-packages/mpd.py", line 248, in _read_pair
    line = self._read_line()
  File "/usr/lib/python3.2/site-packages/mpd.py", line 229, in _read_line
    line = self._rfile.readline()
  File "/usr/lib/python3.2/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 6249: ordinal not in range(128)

I am using mpd 0.17.0, Python 3.2.3 and the latest python-mpd2 from today.

[pfrenssen]

Forgot to mention, when I run it in Python2 it works fine.

[Mic92]

Could you give me the complete output of this script using python2?

[pfrenssen]

Sure, I have put up an archive of the script and the output of the script in both Python 2 and 3 here: http://www.frenssen.be/mpd-test.tar.gz

[Mic92]

It seems to breaks here: {'file': 'Reggae - Ragga - Ska/Sean Paul/Dutty Rock/22 Punkie (Espa\xc3\xb1ol).mp3'} Because the exception stated: UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 6249: ordinal not in range(128)

Currently I can't reproduce it by just naming a file like this. Maybe you could give me this file. The following informations could be also interesting:

import sys,locale
print(sys.getdefaultencoding())
print(sys.getfilesystemencoding())
print(sys.platform)
print(locale.getlocale())

[pfrenssen]

Ah good find, I could not open this file with any other programs either, and the special character in the filename showed up as garbled text in directory listings. Turns out I had no locale enabled with UTF-8 support on this system, I was using the "C" locale. Strange that I have not come across this earlier, I am already using this system for several weeks.

I suppose it is not python-mpd2's responsibility to check that users have their locales configured correctly, so I'm going to close this issue. Thanks very much for your support!

[Mic92]

Ok, now I can reproduce it. But it would be cool anyway, if it would work under this condition.