search

Duke-GCB / DukeDSClient

Command line program to allow uploading, downloading, and managing projects in the duke-data-service.
MIT License
5 stars 6 forks source link

Name or service not known error when uploading #253

Closed johnbradley closed 5 years ago

johnbradley commented 5 years ago

A user reported an error when uploading a few 20G files:

Traceback (most recent call last):
  File "/nfs/software/helmod/apps/Core/ddsclient/2.3.1-gcb01/lib/python2.7/site-packages/ddsc/core/fileuploader.py", line 336, in upload_async
    sender.send()
  File "/nfs/software/helmod/apps/Core/ddsclient/2.3.1-gcb01/lib/python2.7/site-packages/ddsc/core/fileuploader.py", line 379, in send
    self._send_chunk(chunk, chunk_num)
  File "/nfs/software/helmod/apps/Core/ddsclient/2.3.1-gcb01/lib/python2.7/site-packages/ddsc/core/fileuploader.py", line 390, in _send_chunk
    url_info = self.upload_operations.create_file_chunk_url(self.upload_id, chunk_num, chunk)
  File "/nfs/software/helmod/apps/Core/ddsclient/2.3.1-gcb01/lib/python2.7/site-packages/ddsc/core/fileuploader.py", line 173, in create_file_chunk_url
    resp = retry_until_resource_is_consistent(func, self.waiting_monitor)
  File "/nfs/software/helmod/apps/Core/ddsclient/2.3.1-gcb01/lib/python2.7/site-packages/ddsc/core/ddsapi.py", line 1193, in retry_until_resource_is_consistent
    resp = func()
  File "/nfs/software/helmod/apps/Core/ddsclient/2.3.1-gcb01/lib/python2.7/site-packages/ddsc/core/fileuploader.py", line 171, in func
    hash_data.value, hash_data.alg)
  File "/nfs/software/helmod/apps/Core/ddsclient/2.3.1-gcb01/lib/python2.7/site-packages/ddsc/core/ddsapi.py", line 578, in create_upload_url
    return self._put("/uploads/" + upload_id + "/chunks", data)
  File "/nfs/software/helmod/apps/Core/ddsclient/2.3.1-gcb01/lib/python2.7/site-packages/ddsc/core/ddsapi.py", line 41, in retry_function
    result = func(*args, **kwds)
  File "/nfs/software/helmod/apps/Core/ddsclient/2.3.1-gcb01/lib/python2.7/site-packages/ddsc/core/ddsapi.py", line 265, in _put
    resp = self.http.put(url, data_str, headers=headers)
  File "/nfs/software/helmod/apps/Core/ddsclient/2.3.1-gcb01/lib/python2.7/site-packages/requests/sessions.py", line 593, in put
    return self.request('PUT', url, data=data, **kwargs)
  File "/nfs/software/helmod/apps/Core/ddsclient/2.3.1-gcb01/lib/python2.7/site-packages/requests/sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
  File "/nfs/software/helmod/apps/Core/ddsclient/2.3.1-gcb01/lib/python2.7/site-packages/requests/sessions.py", line 646, in send
    r = adapter.send(request, **kwargs)
  File "/nfs/software/helmod/apps/Core/ddsclient/2.3.1-gcb01/lib/python2.7/site-packages/requests/adapters.py", line 516, in send
    raise ConnectionError(e, request=request)
ConnectionError: HTTPSConnectionPool(host='api.dataservice.duke.edu', port=443): Max retries exceeded with url: /api/v1/uploads/021f8f04-271a-4af4-a43c-736b0dfb9f5b/chunks (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f15a936d510>: Failed to establish a new connection: [Errno -2] Name or service not known',))

If I'm trying to upload 400 files, it'll usually crash with this error 5-10 times; I restart it each time (and eventually it finishes).