search

Duke-GCB / DukeDSClient

Command line program to allow uploading, downloading, and managing projects in the duke-data-service.
MIT License
5 stars 6 forks source link

ReadTimeout during upload - creating chunk url #266

Closed johnbradley closed 5 years ago

johnbradley commented 5 years ago

A user reported an error when running multiple ddsclient uploads at the same time.

One failed with the following error:

File "/opt/virtualenv/datadelivery/lib/python2.7/site-packages/ddsc/core/fileuploader.py", line 402, in _send_chunk
    url_info = self.upload_operations.create_file_chunk_url(self.upload_id, chunk_num, chunk)
  File "/opt/virtualenv/datadelivery/lib/python2.7/site-packages/ddsc/core/fileuploader.py", line 179, in create_file_chunk_url
    resp = retry_until_resource_is_consistent(func, self.waiting_monitor)
  File "/opt/virtualenv/datadelivery/lib/python2.7/site-packages/ddsc/core/ddsapi.py", line 1211, in retry_until_resource_is_consistent
    resp = func()
  File "/opt/virtualenv/datadelivery/lib/python2.7/site-packages/ddsc/core/fileuploader.py", line 177, in func
    hash_data.value, hash_data.alg)
  File "/opt/virtualenv/datadelivery/lib/python2.7/site-packages/ddsc/core/ddsapi.py", line 596, in create_upload_url
    return self._put("/uploads/" + upload_id + "/chunks", data)
  File "/opt/virtualenv/datadelivery/lib/python2.7/site-packages/ddsc/core/ddsapi.py", line 47, in retry_function
    result = func(*args, **kwds)
  File "/opt/virtualenv/datadelivery/lib/python2.7/site-packages/ddsc/core/ddsapi.py", line 283, in _put
    resp = self.http.put(url, data_str, headers=headers)
  File "/opt/virtualenv/datadelivery/lib/python2.7/site-packages/requests/sessions.py", line 593, in put
    return self.request('PUT', url, data=data, **kwargs)
  File "/opt/virtualenv/datadelivery/lib/python2.7/site-packages/requests/sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
  File "/opt/virtualenv/datadelivery/lib/python2.7/site-packages/requests/sessions.py", line 646, in send
    r = adapter.send(request, **kwargs)
  File "/opt/virtualenv/datadelivery/lib/python2.7/site-packages/requests/adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
ReadTimeout: HTTPSConnectionPool(host='api.dataservice.duke.edu', port=443): Read timed out. (read timeout=None)

The error occurred requesting a pre-signed URL to upload a file chunk via the DukeDS API PUT /api/v1/uploads/{id}/chunks endpoint