Unicode error during download

[GoogleCodeExporter]

When trying to download a song called "Im Fruehling" (replace ue with u 
umlaut), pymazon errors out with the following stack trace (ubuntu maverick, 
latest pymazon on pypi):

Exception in thread Thread-2:
Traceback (most recent call last):
  File "/usr/lib/python2.6/threading.py", line 532, in __bootstrap_inner
    self.run()
  File "/home/smulloni/.virtualenvs/pymazon/lib/python2.6/site-packages/pymazon/core/downloader.py", line 54, in run
    self.do_work()
  File "/home/smulloni/.virtualenvs/pymazon/lib/python2.6/site-packages/pymazon/core/downloader.py", line 59, in do_work
    handle = self._connect(obj)
  File "/home/smulloni/.virtualenvs/pymazon/lib/python2.6/site-packages/pymazon/core/downloader.py", line 83, in _connect
    handle = self.opener.open(request)
  File "/usr/lib/python2.6/urllib2.py", line 391, in open
    response = self._open(req, data)
  File "/usr/lib/python2.6/urllib2.py", line 409, in _open
    '_open', req)
  File "/usr/lib/python2.6/urllib2.py", line 369, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.6/urllib2.py", line 1170, in http_open
    return self.do_open(httplib.HTTPConnection, req)
  File "/usr/lib/python2.6/urllib2.py", line 1142, in do_open
    h.request(req.get_method(), req.get_selector(), req.data, headers)
  File "/usr/lib/python2.6/httplib.py", line 914, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib/python2.6/httplib.py", line 951, in _send_request
    self.endheaders()
  File "/usr/lib/python2.6/httplib.py", line 908, in endheaders
    self._send_output()
  File "/usr/lib/python2.6/httplib.py", line 780, in _send_output
    self.send(msg)
  File "/usr/lib/python2.6/httplib.py", line 759, in send
    self.sock.sendall(str)
  File "<string>", line 1, in sendall
UnicodeEncodeError: 'ascii' codec can't encode character u'\xfc' in position 
1393: ordinal not in range(128)

Original issue reported on code.google.com by jsmull...@gmail.com on 15 Apr 2011 at 8:57

[GoogleCodeExporter]

Ugh, why does amazon use non-ascii characters in the urls???!!!!????

Original comment by SCColbert@gmail.com on 15 Apr 2011 at 9:29

• Changed state: Accepted

[GoogleCodeExporter]

I attach a trivial diff which fixes the problem.  The url needs to be encoded 
as a utf-8 string before the request is issued.

Original comment by jsmull...@gmail.com on 15 Apr 2011 at 9:29

Attachments:

• pymazon-unicode.patch

[GoogleCodeExporter]

Thanks! The patch in comment 2 worked. I manually edited the downloader.py file 
myself and now files with unicode characters download just fine!

Original comment by yzhern...@gmail.com on 29 Nov 2011 at 6:16

[GoogleCodeExporter]

The patch in comment 2 fixed an undownloadable tune for me too. Looks like 
either a plain bug in python 2.7.2, or a clash between the design of Python 2 
Unicode handling with respect to urls, and Amazon's url-encoding practices, I 
have no idea which.

Original comment by catmat...@gmail.com on 27 Mar 2012 at 3:37

[GoogleCodeExporter]

It's not just amazon's URLs. There's another similar problem when you try and 
download a track whose name has non-ascii characters in its name, and pymazon 
tries to present you with a list for confirmation:

Traceback (most recent call last):
  File "/usr/bin/pymazon", line 67, in <module>
    main()
  File "/usr/bin/pymazon", line 37, in main
    main(amzs)
  File "/usr/lib/python2.7/site-packages/pymazon/cmd/ui.py", line 134, in main
    runner.run()
  File "/usr/lib/python2.7/site-packages/pymazon/cmd/ui.py", line 65, in run
    print line.encode('ascii')
UnicodeEncodeError: 'ascii' codec can't encode character u'\xdf' in position 
25: ordinal not in range(128)

Then, later, when it tries to save the files:

Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 551, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/site-packages/pymazon/core/downloader.py", line 54, in run
    self.do_work()
  File "/usr/lib/python2.7/site-packages/pymazon/core/downloader.py", line 68, in do_work
    obj.save(data)
  File "/usr/lib/python2.7/site-packages/pymazon/core/item_model.py", line 133, in save
    super(Track, self).save(save_path, data)
  File "/usr/lib/python2.7/site-packages/pymazon/core/item_model.py", line 77, in save
    fname = self._safe_save_name(fname)
  File "/usr/lib/python2.7/site-packages/pymazon/core/item_model.py", line 70, in _safe_save_name
    if not os.path.isfile(nfname):
  File "/usr/lib/python2.7/genericpath.py", line 29, in isfile
    st = os.stat(path)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xfc' in position 
17: ordinal not in range(128)

Similar tiny patches to the one mentioned above fix the problems at those lines 
too.

Original comment by joel.gl...@gmail.com on 18 May 2012 at 10:21