SSL related connection errors when uploading files

[mad4linux]

Hello Tom First of all, thanks for your great work creating this python library. It will hopefully save me a lot of hassle in the future with uploading my backups to pcloud. I'm trying to upload a bunch of files to pcloud (a few hundred). For this, I'm generating a list of file names and upload these as soon as I have 11 files in the list. I'm using, a line like this to upload my files: cloudrive.uploadfile(folderid=bBSID, files=uploadBuffer) uploadBuffer is a list of file paths. This works fine for a while, but sooner or later I'm getting the follwoing error and the script stops: `Traceback (most recent call last): File "/usr/lib/python3/dist-packages/urllib3/contrib/pyopenssl.py", line 317, in _send_until_done return self.connection.send(data) File "/usr/lib/python3/dist-packages/OpenSSL/SSL.py", line 1737, in send self._raise_ssl_error(self._ssl, result) File "/usr/lib/python3/dist-packages/OpenSSL/SSL.py", line 1639, in _raise_ssl_error raise SysCallError(errno, errorcode.get(errno)) OpenSSL.SSL.SysCallError: (104, 'ECONNRESET')

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 600, in urlopen chunked=chunked) File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 354, in _make_request conn.request(method, url, **httplib_request_kw) File "/usr/lib/python3.7/http/client.py", line 1260, in request self._send_request(method, url, body, headers, encode_chunked) File "/usr/lib/python3.7/http/client.py", line 1306, in _send_request self.endheaders(body, encode_chunked=encode_chunked) File "/usr/lib/python3.7/http/client.py", line 1255, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "/usr/lib/python3.7/http/client.py", line 1069, in _send_output self.send(chunk) File "/usr/lib/python3.7/http/client.py", line 991, in send self.sock.sendall(data) File "/usr/lib/python3/dist-packages/urllib3/contrib/pyopenssl.py", line 328, in sendall sent = self._send_until_done(data[total_sent:total_sent + SSL_WRITE_BLOCKSIZE]) File "/usr/lib/python3/dist-packages/urllib3/contrib/pyopenssl.py", line 323, in _send_until_done raise SocketError(str(e)) OSError: (104, 'ECONNRESET')

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/lib/python3/dist-packages/requests/adapters.py", line 449, in send timeout=timeout File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 638, in urlopen _stacktrace=sys.exc_info()[2]) File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 367, in increment raise six.reraise(type(error), error, _stacktrace) File "/usr/lib/python3/dist-packages/six.py", line 692, in reraise raise value.with_traceback(tb) File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 600, in urlopen chunked=chunked) File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 354, in _make_request conn.request(method, url, **httplib_request_kw) File "/usr/lib/python3.7/http/client.py", line 1260, in request self._send_request(method, url, body, headers, encode_chunked) File "/usr/lib/python3.7/http/client.py", line 1306, in _send_request self.endheaders(body, encode_chunked=encode_chunked) File "/usr/lib/python3.7/http/client.py", line 1255, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "/usr/lib/python3.7/http/client.py", line 1069, in _send_output self.send(chunk) File "/usr/lib/python3.7/http/client.py", line 991, in send self.sock.sendall(data) File "/usr/lib/python3/dist-packages/urllib3/contrib/pyopenssl.py", line 328, in sendall sent = self._send_until_done(data[total_sent:total_sent + SSL_WRITE_BLOCKSIZE]) File "/usr/lib/python3/dist-packages/urllib3/contrib/pyopenssl.py", line 323, in _send_until_done raise SocketError(str(e)) urllib3.exceptions.ProtocolError: ('Connection aborted.', OSError("(104, 'ECONNRESET')"))

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap self.run() File "/usr/lib/python3.7/multiprocessing/process.py", line 99, in run self._target(*self._args, *self._kwargs) File "./pCloudBackup.py", line 112, in uploadFiles cloudrive.uploadfile(folderid=bBSID, files=uploadBuffer) File "/usr/local/lib/python3.7/dist-packages/pcloud/validate.py", line 19, in wrapper return func(args, kwargs) File "/usr/local/lib/python3.7/dist-packages/pcloud/api.py", line 156, in uploadfile return self._upload("uploadfile", files, kwargs) File "/usr/local/lib/python3.7/dist-packages/pcloud/api.py", line 135, in _upload resp = self.session.post(self.endpoint + method, files=files, data=kwargs) File "/usr/lib/python3/dist-packages/requests/sessions.py", line 581, in post return self.request('POST', url, data=data, json=json, kwargs) File "/usr/lib/python3/dist-packages/requests/sessions.py", line 533, in request resp = self.send(prep, send_kwargs) File "/usr/lib/python3/dist-packages/requests/sessions.py", line 646, in send r = adapter.send(request, **kwargs) File "/usr/lib/python3/dist-packages/requests/adapters.py", line 498, in send raise ConnectionError(err, request=request) requests.exceptions.ConnectionError: ('Connection aborted.', OSError("(104, 'ECONNRESET')")) `

The script is running on a debian server with python 3.7.3 I can also send you the full code of my script, if this would help you.

[tomgross]

Hi @mad4linux Glad you like the library. This issue comes vom the requests-library pycloud uses. There is an explaination here: https://github.com/psf/requests/issues/2543#issuecomment-94648488 The workaround they suggest is a try except clause to fail gracefully.

Are you running multiple threads simultaniously? Could this be a reason pCloud blocks you temporarily?

[mad4linux]

Hi Tom Thank you for the quick reply. So the problem is coming from the server side and it's not a library bug. I'm actually running multiple threads, but only one is uploading files. I will catch the error and reupload as a work around. At the moment, I'm sending a bunch of sometimes very small files, so maybe the server does not like this many connections in such a short time. I certainly will have to optimise my code.

[mad4linux]

As it turned out, the cause for the connection errors have been special characters in filenames. Stripping all special characters from file names solved the problem. A simple way to do this is: filenamestr.encode("ascii", "ignore").decode()