dropbox / dropbox-sdk-python

The Official Dropbox API V2 SDK for Python
https://www.dropbox.com/developers
MIT License
932 stars 317 forks source link

EOF occurred in violation of protocol #432

Closed toroConverter closed 2 years ago

toroConverter commented 2 years ago

Hi, I'm using this library to upload some files to my Dropbox. Very often I got this error:

HTTPSConnectionPool(host='content.dropboxapi.com', port=443): Max retries exceeded with url: /2/files/upload_session/append_v2 (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:2396)')))

Here the code I'm using to upload my files:

def upload(self, file_path, file_row_index, target_path='/', chunk_size=60):
    file_size = os.path.getsize(file_path)
    target_path += os.path.basename(file_path)
    chunk_size = chunk_size * 1024 * 1024
    uploaded_size = 0

    with open(file_path, "rb") as f:
        if file_size <= chunk_size:
            self.dbx.files_upload(f.read(), target_path)
        else:
            upload_session_start_result = self.dbx.files_upload_session_start(f.read(chunk_size))
            uploaded_size += chunk_size

            cursor = dropbox.files.UploadSessionCursor(session_id=upload_session_start_result.session_id,
                                                       offset=f.tell())
            commit = dropbox.files.CommitInfo(path=target_path)

            while f.tell() < file_size:
                if (file_size - f.tell()) <= chunk_size:
                    self.dbx.files_upload_session_finish(f.read(chunk_size), cursor, commit)
                    uploaded_size = file_size
                else:
                    self.dbx.files_upload_session_append_v2(f.read(chunk_size), cursor)
                    cursor.offset = f.tell()
                    uploaded_size += chunk_size

This function is inside a class where I have the dbx variable created by calling dropbox.Dropbox(TOKEN), where token is my temporary access token generated with refresh token. The files that I'm uploading are very often more than 150mb because with the ones less than 150 I have no problem.

Versions Pip freeze output:

altgraph==0.17.2
certifi==2021.10.8
cffi==1.15.0
charset-normalizer==2.0.12
colorama==0.4.4
cryptography==37.0.2
dropbox==11.31.0
future==0.18.2
idna==3.3
natsort==8.1.0
ndg-httpsclient==0.5.1
pefile==2021.9.3
playsound==1.3.0
ply==3.11
pyasn1==0.4.8
pycparser==2.21
pyinstaller==5.1
pyinstaller-hooks-contrib==2022.4
pyOpenSSL==22.0.0
PySide6==6.3.0
PySide6-Addons==6.3.0
PySide6-Essentials==6.3.0
pysubs2==1.4.2
pywin32-ctypes==0.2.0
requests==2.27.1
shiboken6==6.3.0
six==1.16.0
stone==3.3.1
tqdm==4.64.0
urllib3==1.26.9

Python 3.10.4 Windows Server 2019

Hope that this information will help you in resolving my problem. In case not, please let me know and I will provide as much information i can ;)

greg-db commented 2 years ago

Thanks for the report. Can you let me know when you started seeing this issue?

Also, you mentioned you see this "Very often". Do you see this for every single files_upload_session_append_v2 call, or only some? If it's just some, please let me know about what percent fail.

And do you see this for files_upload_session_start and files_upload_session_finish as well?

In any case, I don't see anything wrong in the code, and I just tried this code myself and the issue doesn't reproduce for me. Is there anything in your network connection, such as a proxy, firewall, VPN, etc. that may be interfering with these connections?

toroConverter commented 2 years ago

Thanks for the report. Can you let me know when you started seeing this issue?

I'm not sure 100%, but after the latest update it happens always. The previous week i manage to upload very large file (30/40gb each) with no error.

Also, you mentioned you see this "Very often". Do you see this for every single files_upload_session_append_v2 call, or only some? If it's just some, please let me know about what percent fail.

Yes, at this moment every files_upload_session_append_v2 fails.

And do you see this for files_upload_session_start and files_upload_session_finish as well?

No issue a the moment.

In any case, I don't see anything wrong in the code, and I just tried this code myself and the issue doesn't reproduce for me. Is there anything in your network connection, such as a proxy, firewall, VPN, etc. that may be interfering with these connections?

The connection is the same as last week, so no particular VPN or firewall. The only thing that i forgot to mention is that this class is used to create some threads (4 a the moment) to upload consequently more file at the same time.

Since there are no error in my code, I will try to restart my computer and see if it was a temporary issue. If it will not work I will try to downgrade to previous version and then I will update you.

toroConverter commented 2 years ago

Hi @greg-db, now everything works fine. Probably my computer need a restart. Anyway sorry for bothering you and thanks for your help ;)