Upload script failing with UnicodeDecodeError

[silvermike]

I ran into this problem with the api while trying to upload videos and can reproduce it with vimeo-uploadv2.py

Every video I try to upload fails with a UnicodeDecodeError like:

File "c:\foobar\tools\vimeo\convenience.py", line 111, in upload self._post_to_endpoint(open(file_path)) File "c:\foobar\tools\vimeo\convenience.py", line 94, in _post_to_endpoint headers=headers) File "c:\foobar\tools\vimeo\httplib2wrap\__init__.py", line 34, in request_with_files headers, *args, **kwargs) File "c:\foobar\tools\httplib2\__init__.py", line 1584, in request (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey) File "c:\foobar\tools\httplib2\__init__.py", line 1332, in _request (response, content) = self._conn_request(conn, request_uri, method, body, headers) File "c:\foobar\tools\httplib2\__init__.py", line 1269, in _conn_request conn.request(method, request_uri, body, headers) File "C:\Python27\lib\httplib.py", line 958, in request self._send_request(method, url, body, headers) File "C:\Python27\lib\httplib.py", line 992, in _send_request self.endheaders(body) File "C:\Python27\lib\httplib.py", line 954, in endheaders self._send_output(message_body) File "C:\Python27\lib\httplib.py", line 812, in _send_output msg += message_body UnicodeDecodeError: 'ascii' codec can't decode byte 0xb2 in position 1280: ordinal not in range(128)

Environment: python 2.7.3 64 bit Windows 7 64 bit *httplib2 v0.7.5

I dug up this http://bugs.python.org/issue11898 but the conclusion on there is that it is supposed to work.

Any ideas? I tried a couple hacks in the httplib2 wrapper, but couldn't come up with anything that worked.

[dkm]

I dug up this http://bugs.python.org/issue11898 but the conclusion on there is that it is supposed to work.

Any ideas? I tried a couple hacks in the httplib2 wrapper, but couldn't come up with anything that worked.

Hi!

what kind of string are you passing as the title of the video ? It's possible that some part of the code does not like something not ascii. Can you try that ? If this does not work, can you please give more info on how you start the script ?

Thanks

[silvermike]

I’m not trying to set a title. I run the script like:

vimeo-uploadv2.py -k e_6 -s a_7 -t 7a -y 73 -f z:\edits\videos\000.wmv

As a sanity check I did upload the same video through the website successfully.

From: Marc Poulhiès [mailto:notifications@github.com] Sent: Friday, September 07, 2012 1:51 AM To: dkm/python-vimeo Cc: Michael Cirello Subject: Re: [python-vimeo] Upload script failing with UnicodeDecodeError (#9)

I dug up this http://bugs.python.org/issue11898 but the conclusion on there is that it is supposed to work.

Any ideas? I tried a couple hacks in the httplib2 wrapper, but couldn't come up with anything that worked.

Hi!

what kind of string are you passing as the title of the video ? It's possible that some part of the code does not like something not ascii. Can you try that ? If this does not work, can you please give more info on how you start the script ?

Thanks

— Reply to this email directly or view it on GitHubhttps://github.com/dkm/python-vimeo/issues/9#issuecomment-8356658.

[silvermike]

So here's my analysis and (hackish) solution:

The error is coming from this part of httplib.py:

    # If msg and message_body are sent in a single send() call,
    # it will avoid performance problems caused by the interaction
    # between delayed ack and the Nagle algorithm.
    if isinstance(message_body, str):
        msg += message_body

At this point msg is unicode, and our message_body is a str, so it tries to convert message_body to unicode and raises the exception.

I tried to fix it by converting the message_body to a bytearray before sending, thereby skipping the offending line in httplib, but I would just end up with a 708 error from Vimeo and gave up.

My solution - switch to streaming uploads. Two changes -

1) When getting the ticket, set the streaming method:

t = client.vimeo_videos_upload_getTicket(upload_method='streaming')

2) Replace the call to self._post_to_endpoint in the upload function of convenience.py with a call to my streaming upload function:

        import mimetypes
        import os
        import urllib2

        class FileWithLength(file):
            def __init__(self, path):
                self._total = os.path.getsize(path)
                file.__init__(self, path, 'rb')

            def __len__(self):
                return self._total

        file_stream = FileWithLength(file_path)
        type = mimetypes.guess_type(file_path)[0]
        if not type:
            raise VimeoError("Could not guess content type")

        request = urllib2.Request(self.endpoint, file_stream)
        request.add_header('Content-Type', type)
        request.get_method = lambda: 'PUT'
        urllib2.urlopen(request)            

This seems a lot simpler to me, but maybe I'm missing a good reason this is a bad idea..

[dkm]

On 2012-09-13 00:15, silvermike wrote:

So here's my analysis and (hackish) solution:

Thanks for looking into this issue even if I did not answer (I'm a bit busy now...)

This seems a lot simpler to me, but maybe I'm missing a good reason this is a bad idea..

Have you tried on something else than windows if you fix/workaround does not break ? If it's working nicely, I could apply your patch :)

Marc