search

masperro / httplib2

Automatically exported from code.google.com/p/httplib2
0 stars 0 forks source link

UnicodeEncodeError when filename parameter of the Content-Disposition header has non-ascii characters. #138

Closed GoogleCodeExporter closed 8 years ago

GoogleCodeExporter commented 8 years ago 
1. How to reproduce this problem.

$ python3.2
Python 3.2 (r32:88445, Mar  2 2011, 11:23:48) 
[GCC 4.2.1 (Apple Inc. build 5664)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import httplib2
>>> h = httplib2.Http(".cache")
>>> response, content = 
h.request("http://maps.google.com/maps/ms?ie=UTF8&hl=ja&msa=0&output=kml&msid=21
2735829903317511302.00049fc13f56b1b8f1555", "GET")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/msmhrt/local/mypython/3.2/lib/python3.2/site-packages/httplib2/__init__.py", line 1148, in request
    (response, new_content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
  File "/Users/msmhrt/local/mypython/3.2/lib/python3.2/site-packages/httplib2/__init__.py", line 1000, in _request
    _updateCache(headers, response, content, self.cache, cachekey)
  File "/Users/msmhrt/local/mypython/3.2/lib/python3.2/site-packages/httplib2/__init__.py", line 363, in _updateCache
    header_str = info.as_string()
  File "/Users/python/local/3.2/lib/python3.2/email/message.py", line 167, in as_string
    g.flatten(self, unixfrom=unixfrom)
  File "/Users/python/local/3.2/lib/python3.2/email/generator.py", line 88, in flatten
    self._write(msg)
  File "/Users/python/local/3.2/lib/python3.2/email/generator.py", line 141, in _write
    self._write_headers(msg)
  File "/Users/python/local/3.2/lib/python3.2/email/generator.py", line 175, in _write_headers
    header_name=h)
  File "/Users/python/local/3.2/lib/python3.2/email/header.py", line 197, in __init__
    self.append(s, charset, errors)
  File "/Users/python/local/3.2/lib/python3.2/email/header.py", line 275, in append
    s.encode(output_charset, errors)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 22-27: 
ordinal not in range(128)
>>> 

2. Additional information
I'm using Mac OS X 10.6.7, CPython 3.2 and httplib2 tip revision.

$ curl -v 
"http://maps.google.com/maps/ms?ie=UTF8&hl=ja&msa=0&output=kml&msid=212735829903
317511302.00049fc13f56b1b8f1555" 
* About to connect() to maps.google.com port 80 (#0)
*   Trying 66.249.89.104... connected
* Connected to maps.google.com (66.249.89.104) port 80 (#0)
> GET 
/maps/ms?ie=UTF8&hl=ja&msa=0&output=kml&msid=212735829903317511302.00049fc13f56b
1b8f1555 HTTP/1.1
> User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 
OpenSSL/0.9.8l zlib/1.2.3
> Host: maps.google.com
> Accept: */*
> 
< HTTP/1.1 200 OK
< Content-Type: application/vnd.google-earth.kml+xml; charset=UTF-8
< Date: Thu, 31 Mar 2011 07:53:07 GMT
< Expires: -1
< Cache-Control: private, max-age=0
< Content-Disposition: attachment; filename="無題.kml"
< Set-Cookie: 
PREF=ID=d9c22f4d16b0a716:TM=1301557987:LM=1301557987:S=hg-y0oyQvP_Y6a08; 
expires=Sat, 30-Mar-2013 07:53:07 GMT; path=/; domain=.google.com
< X-Content-Type-Options: nosniff
< Server: mfe
< X-XSS-Protection: 1; mode=block
< Transfer-Encoding: chunked
< 
<?xml version="1.0" encoding="UTF-8"?>
<kml xmlns="http://earth.google.com/kml/2.2">
<Document>
  <name>無題</name>
  <description><![CDATA[]]></description>
</Document>
</kml>
* Connection #0 to host maps.google.com left intact
* Closing connection #0
$

Original issue reported on code.google.com by msm...@gmail.com on 31 Mar 2011 at 8:20

GoogleCodeExporter commented 8 years ago 
I can't reproduce this, here's my output:

>>> h = httplib2.Http('.cache')
>>> response, content = 
h.request("http://maps.google.com/maps/ms?ie=UTF8&hl=ja&msa=0&output=kml&msid=21
2735829903317511302.00049fc13f56b1b8f1555", "GET")
>>> content
'<?xml version="1.0" encoding="UTF-8"?>\n<kml 
xmlns="http://earth.google.com/kml/2.2">\n<Document>\n  
<name>\xe7\x84\xa1\xe9\xa1\x8c</name>\n  
<description><![CDATA[]]></description>\n</Document>\n</kml>\n'
>>> response
{'status': '200', 'x-xss-protection': '1; mode=block', 
'x-content-type-options': 'nosniff', 'content-disposition': 'attachment; 
filename="\xe7\x84\xa1\xe9\xa1\x8c.kml"', 'transfer-encoding': 'chunked', 
'set-cookie': 
'PREF=ID=e36f33a09693f9d1:TM=1301591143:LM=1301591143:S=Aa1xQUC5S2MUDp2J; 
expires=Sat, 30-Mar-2013 17:05:43 GMT; path=/; domain=.google.com', 'expires': 
'-1', 'server': 'mfe', 'cache-control': 'private, max-age=0', 'date': 'Thu, 31 
Mar 2011 17:05:43 GMT', 'content-type': 'application/vnd.google-earth.kml+xml; 
charset=UTF-8', 'content-location': 
'http://maps.google.com/maps/ms?ie=UTF8&hl=ja&msa=0&output=kml&msid=212735829903
317511302.00049fc13f56b1b8f1555'}

Original comment by umb...@gmail.com on 31 Mar 2011 at 5:06

GoogleCodeExporter commented 8 years ago 
I'm using OS X 10.6.6,  Python 2.6.1. May be a Python3 issue.

Original comment by umb...@gmail.com on 31 Mar 2011 at 5:08

GoogleCodeExporter commented 8 years ago 
I have tested on Python 2.7.1 and works correctly.
It seems that this problem only occurs on Python 3.

$ python2.7 
Python 2.7.1 (r271:86832, Dec 23 2010, 22:25:36) 
[GCC 4.2.1 (Apple Inc. build 5664)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import httplib2
>>> h = httplib2.Http(".cache")
>>> response, content = 
h.request("http://maps.google.com/maps/ms?ie=UTF8&hl=ja&msa=0&output=kml&msid=21
2735829903317511302.00049fc13f56b1b8f1555", "GET")
>>> content
'<?xml version="1.0" encoding="UTF-8"?>\n<kml 
xmlns="http://earth.google.com/kml/2.2">\n<Document>\n  
<name>\xe7\x84\xa1\xe9\xa1\x8c</name>\n  
<description><![CDATA[]]></description>\n</Document>\n</kml>\n'
>>> response
{'status': '200', 'x-xss-protection': '1; mode=block', 
'x-content-type-options': 'nosniff', 'content-disposition': 'attachment; 
filename="\xe7\x84\xa1\xe9\xa1\x8c.kml"', 'transfer-encoding': 'chunked', 
'set-cookie': 
'PREF=ID=6e5da26c21ad6ca9:TM=1301612276:LM=1301612276:S=Msq_hdxdikrNVIGF; 
expires=Sat, 30-Mar-2013 22:57:56 GMT; path=/; domain=.google.com', 'expires': 
'-1', 'server': 'mfe', 'cache-control': 'private, max-age=0', 'date': 'Thu, 31 
Mar 2011 22:57:56 GMT', 'content-type': 'application/vnd.google-earth.kml+xml; 
charset=UTF-8', 'content-location': 
'http://maps.google.com/maps/ms?ie=UTF8&hl=ja&msa=0&output=kml&msid=212735829903
317511302.00049fc13f56b1b8f1555'}
>>>

Original comment by msm...@gmail.com on 31 Mar 2011 at 11:09

GoogleCodeExporter commented 8 years ago 
Fixed in 
http://code.google.com/p/httplib2/source/detail?r=8b6bf8972e299e9eeaf411fa25fff2
da97b8c0dc

Original comment by joe.gregorio@gmail.com on 13 Jun 2011 at 4:45

GoogleCodeExporter commented 8 years ago 
email.message_from_bytes() function has been introduced in Python 3.2. Is it 
intentional that Python 3.1 is no longer supported?

(Personally I don't need to use Python 3.1.)

Original comment by Arfrever...@gmail.com on 10 Jul 2011 at 12:46