pf.py failed to upload a file if it contains a funky UTF-8 character

[GoogleCodeExporter]

Steps to Reproduce:

- pf.py put <<attached file>>

Results:

Traceback (most recent call last):
  File "C:\Users\tyip\Dev\Project\pageforestapp\bin\pf.py", line 475, in <module>
    main()
  File "C:\Users\tyip\Dev\Project\pageforestapp\bin\pf.py", line 468, in main
    globals()[options.command + '_command'](args)
  File "C:\Users\tyip\Dev\Project\pageforestapp\bin\pf.py", line 349, in put_command
    upload_file(path)
  File "C:\Users\tyip\Dev\Project\pageforestapp\bin\pf.py", line 193, in upload_file
    response = urllib2.urlopen(PutRequest(url, data))
  File "C:\Python27\lib\urllib2.py", line 126, in urlopen
    return _opener.open(url, data, timeout)
  File "C:\Python27\lib\urllib2.py", line 392, in open
    response = self._open(req, data)
  File "C:\Python27\lib\urllib2.py", line 410, in _open
    '_open', req)
  File "C:\Python27\lib\urllib2.py", line 370, in _call_chain
    result = func(*args)
  File "C:\Python27\lib\urllib2.py", line 1186, in http_open
    return self.do_open(httplib.HTTPConnection, req)
  File "C:\Python27\lib\urllib2.py", line 1155, in do_open
    h.request(req.get_method(), req.get_selector(), req.data, headers)
  File "C:\Python27\lib\httplib.py", line 941, in request
    self._send_request(method, url, body, headers)
  File "C:\Python27\lib\httplib.py", line 975, in _send_request
    self.endheaders(body)
  File "C:\Python27\lib\httplib.py", line 937, in endheaders
    self._send_output(message_body)
  File "C:\Python27\lib\httplib.py", line 795, in _send_output
    msg += message_body
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 5066: 
ordinal not in range(128)

Original issue reported on code.google.com by thomas....@gmail.com on 8 Dec 2010 at 8:02

Attachments:

• ascii_problem.html

[GoogleCodeExporter]

This works for me on Ubuntu.  Maybe a windows issue only with pf.py?

http://pftest.pageforest.com/ascii_problem.html

This file seems identical to the one posted here.  Note that it does contain a 
3-byte Unicode character (E2 86 92).

I did fix a bug where we were not by default settings our results to text/html; 
charset=utf-8.  Now that I've done so, this file displays correctly in the 
browser as well (even in the charset is not set in the html meta header).

Original comment by mckoss@gmail.com on 9 Dec 2010 at 2:47

• Changed state: Fixed

[GoogleCodeExporter]

Newer stack track with Dec 8, 2010 version of pf.py.

---
Traceback (most recent call last):
  File "..\..\bin\pf.py", line 507, in <module>
    main()
  File "..\..\bin\pf.py", line 500, in main
    globals()[options.command + '_command'](args)
  File "..\..\bin\pf.py", line 378, in put_command
    upload_dir(path)
  File "..\..\bin\pf.py", line 289, in upload_dir
    upload_file(os.path.join(dirpath, filename))
  File "..\..\bin\pf.py", line 210, in upload_file
    response = urllib2.urlopen(PutRequest(url, data))
  File "C:\Python27\lib\urllib2.py", line 126, in urlopen
    return _opener.open(url, data, timeout)
  File "C:\Python27\lib\urllib2.py", line 392, in open
    response = self._open(req, data)
  File "C:\Python27\lib\urllib2.py", line 410, in _open
    '_open', req)
  File "C:\Python27\lib\urllib2.py", line 370, in _call_chain
    result = func(*args)
  File "C:\Python27\lib\urllib2.py", line 1186, in http_open
    return self.do_open(httplib.HTTPConnection, req)
  File "C:\Python27\lib\urllib2.py", line 1155, in do_open
    h.request(req.get_method(), req.get_selector(), req.data, headers)
  File "C:\Python27\lib\httplib.py", line 941, in request
    self._send_request(method, url, body, headers)
  File "C:\Python27\lib\httplib.py", line 975, in _send_request
    self.endheaders(body)
  File "C:\Python27\lib\httplib.py", line 937, in endheaders
    self._send_output(message_body)
  File "C:\Python27\lib\httplib.py", line 795, in _send_output
    msg += message_body
UnicodeDecodeError: 'ascii' codec can't decode byte 0xfe in position 0: ordinal 
not in range(128)

Original comment by thomas....@gmail.com on 9 Dec 2010 at 9:40

[GoogleCodeExporter]

Try this pf.py?

Original comment by mckoss@gmail.com on 9 Dec 2010 at 9:44

Attachments:

• pf.py

[GoogleCodeExporter]

That seems to work now...I'll check pf.py in.

Original comment by mckoss@gmail.com on 9 Dec 2010 at 9:46

[GoogleCodeExporter]

Original comment by thomas....@gmail.com on 22 Dec 2010 at 8:57

Attachments:

• 10-medical.png

[GoogleCodeExporter]

Original comment by mckoss@gmail.com on 22 Dec 2010 at 8:57

• Changed state: Accepted

[GoogleCodeExporter]

C:\Users\tyip\Dev\Project\pageforestapp\src\html>pf.py put 
jslib\glyphish\icons\10-medical.png
Application: beedesk-test
Uploading: 
http://admin.beedesk-test.pageforest.com/jslib/glyphish/icons/10-medical.png 
(210 bytes)
Traceback (most recent call last):
  File "C:\Users\tyip\Dev\Project\pageforestapp\src\html\pf.py", line 585, in <module>
    main()
  File "C:\Users\tyip\Dev\Project\pageforestapp\src\html\pf.py", line 578, in main
    globals()[options.command + '_command'](args)
  File "C:\Users\tyip\Dev\Project\pageforestapp\src\html\pf.py", line 438, in put_command
    upload_file(path)
  File "C:\Users\tyip\Dev\Project\pageforestapp\src\html\pf.py", line 228, in upload_file
    response = urllib2.urlopen(PutRequest(url), data)
  File "C:\Python27\lib\urllib2.py", line 126, in urlopen
    return _opener.open(url, data, timeout)
  File "C:\Python27\lib\urllib2.py", line 392, in open
    response = self._open(req, data)
  File "C:\Python27\lib\urllib2.py", line 410, in _open
    '_open', req)
  File "C:\Python27\lib\urllib2.py", line 370, in _call_chain
    result = func(*args)
  File "C:\Python27\lib\urllib2.py", line 1186, in http_open
    return self.do_open(httplib.HTTPConnection, req)
  File "C:\Python27\lib\urllib2.py", line 1155, in do_open
    h.request(req.get_method(), req.get_selector(), req.data, headers)
  File "C:\Python27\lib\httplib.py", line 941, in request
    self._send_request(method, url, body, headers)
  File "C:\Python27\lib\httplib.py", line 975, in _send_request
    self.endheaders(body)
  File "C:\Python27\lib\httplib.py", line 937, in endheaders
    self._send_output(message_body)
  File "C:\Python27\lib\httplib.py", line 795, in _send_output
    msg += message_body
UnicodeDecodeError: 'ascii' codec can't decode byte 0x89 in position 0: ordinal 
not in range(128)

Original comment by thomas....@gmail.com on 22 Dec 2010 at 9:13

[GoogleCodeExporter]

Now encode all uploads in pf.py using base64.

Original comment by mckoss@gmail.com on 22 Dec 2010 at 11:26

• Changed state: Fixed