UTF-8 metadata support

[miksir]

On mopidy local scan I got error: "You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings (like text_factory = str). It is highly recommended that you instead just switch your application to Unicode strings."

[miksir]

Oh, well, not enough.

  File "/usr/share/mopidy/mopidy/mpd/protocol/__init__.py", line 156, in validate
    return func(**callargs)
  File "/usr/share/mopidy/mopidy/mpd/protocol/music_db.py", line 413, in lsinfo
    for path, lookup_future in context.browse(uri, recursive=False):
  File "/usr/share/mopidy/mopidy/mpd/dispatcher.py", line 320, in browse
    path = '/'.join([base_path, ref.name.replace('/', '')])
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 1: ordinal not in range(128)

Probably need to change URI create part...

[tkem]

Could you please provide more information on the original error, i.e. the file that caused the error during local scan? Mopidy uses unicode throughout, so your change would probably break a lot of other stuff.

[tkem]

Hint: running mopidy -v local scan should provide additional debug output.

[miksir]

Full error is

ERROR    2014-09-08 01:14:49,930 [5376:MainThread] mopidy.__main__
  You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings (like text_factory = str). It is highly recommended that you instead just switch your application to Unicode strings.
Traceback (most recent call last):
  File "/usr/share/mopidy/mopidy/__main__.py", line 127, in main
    return args.command.run(args, proxied_config)
  File "/usr/share/mopidy/mopidy/local/commands.py", line 122, in run
    library.add(track)
  File "/usr/lib/python2.7/dist-packages/mopidy_local_sqlite/library.py", line 137, in add
    schema.insert_track(self._connect(), track)
  File "/usr/lib/python2.7/dist-packages/mopidy_local_sqlite/schema.py", line 329, in insert_track
    track.last_modified
ProgrammingError: You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings (like text_factory = str). It is highly recommended that you instead just switch your application to Unicode strings.
Traceback (most recent call last):
  File "/usr/bin/mopidy", line 9, in <module>
    load_entry_point('Mopidy==0.19.4', 'console_scripts', 'mopidy')()
  File "/usr/share/mopidy/mopidy/__main__.py", line 127, in main
    return args.command.run(args, proxied_config)
  File "/usr/share/mopidy/mopidy/local/commands.py", line 122, in run
    library.add(track)
  File "/usr/lib/python2.7/dist-packages/mopidy_local_sqlite/library.py", line 137, in add
    schema.insert_track(self._connect(), track)
  File "/usr/lib/python2.7/dist-packages/mopidy_local_sqlite/schema.py", line 329, in insert_track
    track.last_modified
sqlite3.ProgrammingError: You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings (like text_factory = str). It is highly recommended that you instead just switch your application to Unicode strings.

File name was with following sequence (in hex): 2b d0 b9 2d c2 a6 Sure, I can rename file, but fatal error is no good in this case.

[miksir]

Some utf-8 encoded filenames lead to the same result. For example %D0%90%20%D0%BC%D0%BE%D0%B6%D0%B5%D1%82%20%D0%B1%D1%8B%D1%82%D1%8C%20%D0%B2%D0%BE%D1%80%D0%BE%D0%BD%D0%B0.mp3

[miksir]

With text_factory = str all scanning fine. About this error - need more time, can't provide more information yet.

[tkem]

When you say "fatal error", do you imply that mopidy stopped scanning at this point? IMO it should just skip this track, and continue with the next.

[tkem]

Thanks for the extra information. Looks like there is an issue when the media file does not contain proper tags and the file name contains non-ASCII characters. Will be tracked in #19.

Thanks for reporting, but please try to provide more detailed bug reports in the first place!

[tkem]

About the "fatal error": this has been confirmed and will be tracked as #20. Thanks for pointing this out.