search

vimeo / vimeo.py

Official Python library for the Vimeo API.
https://developer.vimeo.com
Apache License 2.0
210 stars 84 forks source link

TusCommunicationError when uploading a video of 500MB #141

Open Kinozuko opened 5 years ago

Kinozuko commented 5 years ago

Hi, i have an issue when i try to upload videos (500MB as minimum), at begin is uploading correctly, after a while i got an exepcion, it work perfectly when i upload small video (4MB), i'm using python-sdk with the function upload inside my class, in the construct i initialize the user

`class VimeoAPI():

def upload_file(self,file_path,file_name,file_description):
    file = self._service.upload(file_path,
        data = {
            "name" : file_name,
            "description" : file_description,
            "privacy.add" : False,
            "privacy.comments" : "nobody",
            "privacy.download" : False,
            "privacy.embed" : "private",
            "privacy.views" : "unlisted",
            "chunk_size" : (100 * 1024 * 1024) # 100 MB
        }
    )
    return {
        "response" : True if file else False,   
        "video_uri" : file
    }
`

Python-sdk only use the "tus" option to upload video (when using the upload function inside the Pyvimeo), i'm getting errors based in the tus-py-client, im gettin this exception from tusclient inside exception.py in the vimeo-sdk:

message = getattr(response, 'message') AttributeError: 'TusCommunicationError' object has no attribute 'message'

The exception class is this:

class TusUploadFailed(TusCommunicationError):

"""Should be raised when an attempted upload fails"""
pass

How you can see, the exception is empty and i'm getting that exception, also, i'm getting this:

raise TusCommunicationError(error) tusclient.exceptions.TusCommunicationError: HTTPSConnectionPool(host='files.tus.vimeo.com', port=443): Max retries exceeded with url: /files/vimeo-prod-src-tus-us/8c77eeeb2a0ec1aa466cf045d9c042d6 (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f1e52422a90>: Failed to establish a new connection: [Errno 110] Connection timed out',)) ############################### ###############################

raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='files.tus.vimeo.com', port=443): Max retries exceeded with url: /files/vimeo-prod-src-tus-us/8c77eeeb2a0ec1aa466cf045d9c042d6 (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f1e52422a90>: Failed to establish a new connection: [Errno 110] Connection timed out',)) ############################### ###############################

raise MaxRetryError(_pool, url, error or ResponseError(cause)) urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='files.tus.vimeo.com', port=443): Max retries exceeded with url: /files/vimeo-prod-src-tus-us/8c77eeeb2a0ec1aa466cf045d9c042d6 (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f1e52422a90>: Failed to establish a new connection: [Errno 110] Connection timed out',)) ############################### ###############################

"Failed to establish a new connection: %s" % e) urllib3.exceptions.NewConnectionError: <urllib3.connection.VerifiedHTTPSConnection object at 0x7f1e52422a90>: Failed to establish a new connection: [Errno 110] Connection timed out ############################### ###############################

raise TusUploadFailed(e) tusclient.exceptions.TusUploadFailed: (55, 'SSL_write() returned SYSCALL, errno = 32') ############################### ###############################

pycurl.error: (55, 'SSL_write() returned SYSCALL, errno = 32') ############################### ###############################

raise TusUploadFailed(e) tusclient.exceptions.TusUploadFailed: (55, 'SSL_write() returned SYSCALL, errno = 32') ############################### ###############################

raise TusUploadFailed(e) tusclient.exceptions.TusUploadFailed: (55, 'SSL_write() returned SYSCALL, errno = 32') ############################### ###############################

self.handle.perform() pycurl.error: (55, 'SSL_write() returned SYSCALL, errno = 32') the error is based in the python vimeo-sdk or the tus-py-client or is something else?

erunion commented 5 years ago

Are you receiving this error on every 500MB+ upload, or just received it once? I'm having trouble replicating, and our upload backend team hasn't seen any issues today.

Kinozuko commented 5 years ago

Yes, i tried to upload an .avi and .MKV and got the same error in both upload

erunion commented 5 years ago

What version of Python are you running? What version of the PyVimeo?

Kinozuko commented 5 years ago

Python==3.6.3 and PyVimeo==1.0.11

erunion commented 5 years ago

We're working on an exclusive Python 3 version this week (#140), but I'm curious if this is a problem with your environment and tuspy 0.2.1. Are you able at all to run your upload integration under Python 2?

If not, hold on a couple days until we're able to release the 2.0.0 release that targets just Python 3+ and latest versions of tuspy, and see if that helps.

Kinozuko commented 5 years ago

Never tried with Python 2, all my virtualenv is setup for Python 3, this is all my dependecies for PyVimeo:

`PyVimeo==1.0.11

I will try to run using Python 2 to see if it works

Kinozuko commented 5 years ago

Hi, i tried to run using Python 2 and got this error:

'Unexpected error when uploading through tus.' VideoUploadFailure: (55, 'SSL_write() returned SYSCALL, errno = 32')

xtsimpouris commented 5 years ago

Where do we stand with this? I just got the same exception Unable to upload video to Vimeo ('TusUploadFailed' object has no attribute 'message') with a video of around 60mb. My depedencies in my virtualenv environment are (Python 3):

cachetools==3.0.0
certifi==2018.11.29
chardet==3.0.4
cssmin==0.2.0
Django==2.1.4
django-appconf==1.0.2
django-common-helpers==0.9.2
django-compressor==2.2
django-cron==0.5.1
django-rosetta==0.9.0
django-widget-tweaks==1.4.3
ffmpeg-python==0.1.17
fpdf==1.7.2
future==0.17.1
google-api-python-client==1.7.7
google-auth==1.6.2
google-auth-httplib2==0.0.3
google-auth-oauthlib==0.2.0
gunicorn==19.9.0
htmlmin==0.1.12
httplib2==0.12.0
idna==2.8
markdown2==2.3.7
mysqlclient==1.3.14
numpy==1.16.2
oauth2client==4.1.3
oauthlib==2.1.0
opencv-python==4.0.0.21
Pillow==5.3.0
polib==1.1.0
progressbar2==3.39.2
pyasn1==0.4.5
pyasn1-modules==0.2.3
pycurl==7.43.0
python-utils==2.3.0
pytz==2018.7
PyVimeo==1.0.11
rcssmin==1.0.6
requests==2.21.0
requests-oauthlib==1.0.0
rjsmin==1.0.12
rsa==4.0
six==1.10.0
smsapi-client==2.2.10
tinydb==3.13.0
tuspy==0.2.1
uritemplate==3.0.0
urllib3==1.24.1
uWSGI==2.0.17.1
Kinozuko commented 5 years ago

Never got a solution using the Tus upload option. I used the form upload option, was getting the action link from the form and used a request with PUT to upload large size videos, tested with videos 500MB+ and worked.

ctsimpouris commented 5 years ago

For some reason, vimeo module has a deffault chunk size of 200mb, why?! A single chunk is 200mb? Anyway, I forced with parameter data={'chunk_size': 512 * 1024} down to 512kb. It doesn't matter how small it may be, as internally it is overriden if number of chunks is more than 1024. And now I can upload video through the tus upload option!

deweydb commented 4 years ago

This error appears to be intermittent and an issue with Vimeo's backend. I have received this error before and then when re-attempting with the exact same file and same code at a later date (few hours later) it works fine.

To me this means it has nothing to do with chunk sizes above as suggested, and only a coincidence that it started working again after you changed this setting. Vimeo should actually address this.

Eternal127 commented 4 years ago

I was facing same error and I tried chunk size 1010241024. This is really a tus issue maybe I don't know but it works, I had this issue in some other library which uses tus.