Bug in APEv2 saving of tags with non-ascii characters

[GoogleCodeExporter]

When simply loading and then saving a wavpack file with apev2 tags, in which 
the artist name is Toupé (accented e like that), mutagen fails.

Python 2.7.3 (default, Aug  1 2012, 05:14:39) 
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from mutagen.apev2 import APEv2
>>> a = APEv2("02 Werk Jerk.wv")
>>> a.pprint()
u'Album=Burgers\nAlbum 
Artist=Toup\xe9\nAlbumartistsort=Toup\xe9\nArtist=Toup\xe9\nArtistsort=Toup\xe9\
nAsin=B000AOVKY6\nBarcode=5060039291614\nDisc=1/1\nLabel=Hackpen 
Records\nLanguage=eng\nMUSICBRAINZ_ALBUMSTATUS=official\nMUSICBRAINZ_ALBUMTYPE=a
lbum\nMedia=CD\nMusicbrainz_Albumartistid=694842f9-e679-4292-af79-52a81ffbf2e0\n
Musicbrainz_Albumid=663582ef-53f6-4a0e-b2e3-e7972f5c1ad1\nMusicbrainz_Artistid=6
94842f9-e679-4292-af79-52a81ffbf2e0\nMusicbrainz_Trackid=9cbea852-4179-4035-8a6e
-39b79235477d\nOriginaldate=2005\nReleasecountry=GB\nScript=Latn\nTitle=Werk 
Jerk\nTrack=2/19\nYear=2005\ndate=2005'
>>> a.save()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/mutagen/apev2.py", line 325, in save
    tags = u"".join(tags)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 19: 
ordinal not in range(128)
>>> 

The attached patch seems to fix it, though I'm no Python expert.

Original issue reported on code.google.com by b...@tremby.net on 11 Sep 2012 at 9:45

Attachments:

• patch.diff

[GoogleCodeExporter]

In fact, bizarrely, that fixes it for my command line test case, but when I run 
the program from where I first saw the error (https://bitbucket.org/fk/rgain) I 
still have the issue. I went through its code. It's adding replaygain tags like

tags[u"replaygain_album_gain"] = u"-0.35 dB"

I tried doing this in my test case and sure enough I still get the exception. 
If I remove the u prefixes from the strings, I no longer get the exception. 
Since I don't really understand what's going on with these character encodings 
and string types, I don't know if that's a bug in rgain I'm fixing or if 
there's a problem in mutagen or something else is wrong.

Original comment by b...@tremby.net on 11 Sep 2012 at 10:03

[GoogleCodeExporter]

Can you send me the file or upload it somewhere?

Original comment by reiter.christoph@gmail.com on 11 Sep 2012 at 10:10

[GoogleCodeExporter]

The following is without my above patch. Unsure which if any of these 
exceptions it makes a difference to.

Python 2.7.3 (default, Apr 20 2012, 22:39:59) 
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from mutagen.apev2 import APEv2
>>> a = APEv2("01 Retirement.wv")
>>> a.save()
>>> a[u"test"] = u"testing"
>>> a.save()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/dist-packages/mutagen/apev2.py", line 324, in save
    tags = "".join(tags)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 19: 
ordinal not in range(128)
>>> a["test"] = "testing"
>>> a.save()
>>> a["test"] = u"testing"
>>> a.save()
>>> a[u"test"] = "testing"
>>> a.save()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/dist-packages/mutagen/apev2.py", line 324, in save
    tags = "".join(tags)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 19: 
ordinal not in range(128)
>>> 

Original comment by b...@tremby.net on 11 Sep 2012 at 10:17

Attachments:

• [01 Retirement.wv](https://storage.googleapis.com/google-code-attachments/mutagen/issue-123/comment-3/01 Retirement.wv)

[GoogleCodeExporter]

By the way, that was on a different machine, with mutagen not from SVN, but 
from the ubuntu repository, python-mutagen Version: 1.20-1ubuntu1

Need sleep now, can give more details if needed tomorrow.

Original comment by b...@tremby.net on 11 Sep 2012 at 10:19

[GoogleCodeExporter]

Can you send me the whole file? The attached file doesn't load here.

Original comment by reiter.christoph@gmail.com on 17 Sep 2012 at 6:35

[GoogleCodeExporter]

Thanks. It seems it trips up on unicode keys. APE keys need to be ASCII, so 
just use str keys.

Original comment by reiter.christoph@gmail.com on 17 Sep 2012 at 7:04

• Changed state: Accepted
• Added labels: Type-Defect
• Removed labels: Type-Enhancement

Attachments:

• mutagen-issue-123.wv

[GoogleCodeExporter]

possible fix + test

Original comment by reiter.christoph@gmail.com on 17 Sep 2012 at 9:56

Attachments:

• mutagen-123-v1.patch

[GoogleCodeExporter]

This issue was closed by revision r134.

Original comment by reiter.christoph@gmail.com on 20 Oct 2012 at 10:39

• Changed state: Fixed