esturdivant-usgs / science-base-automation

Automating large USGS ScienceBase data releases
4 stars 2 forks source link

Process times out #65

Open esturdivant-usgs opened 5 years ago

esturdivant-usgs commented 5 years ago
UPLOAD COMPLETED.
Traceback (most recent call last):
  File "//anaconda/envs/sb_py3/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen
    chunked=chunked)
  File "//anaconda/envs/sb_py3/lib/python3.6/site-packages/urllib3/connectionpool.py", line 387, in _make_request
    six.raise_from(e, None)
  File "<string>", line 2, in raise_from
  File "//anaconda/envs/sb_py3/lib/python3.6/site-packages/urllib3/connectionpool.py", line 383, in _make_request
    httplib_response = conn.getresponse()
  File "//anaconda/envs/sb_py3/lib/python3.6/http/client.py", line 1331, in getresponse
    response.begin()
  File "//anaconda/envs/sb_py3/lib/python3.6/http/client.py", line 297, in begin
    version, status, reason = self._read_status()
  File "//anaconda/envs/sb_py3/lib/python3.6/http/client.py", line 266, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "//anaconda/envs/sb_py3/lib/python3.6/site-packages/requests/adapters.py", line 440, in send
    timeout=timeout
  File "//anaconda/envs/sb_py3/lib/python3.6/site-packages/urllib3/connectionpool.py", line 639, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "//anaconda/envs/sb_py3/lib/python3.6/site-packages/urllib3/util/retry.py", line 357, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "//anaconda/envs/sb_py3/lib/python3.6/site-packages/urllib3/packages/six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "//anaconda/envs/sb_py3/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen
    chunked=chunked)
  File "//anaconda/envs/sb_py3/lib/python3.6/site-packages/urllib3/connectionpool.py", line 387, in _make_request
    six.raise_from(e, None)
  File "<string>", line 2, in raise_from
  File "//anaconda/envs/sb_py3/lib/python3.6/site-packages/urllib3/connectionpool.py", line 383, in _make_request
    httplib_response = conn.getresponse()
  File "//anaconda/envs/sb_py3/lib/python3.6/http/client.py", line 1331, in getresponse
    response.begin()
  File "//anaconda/envs/sb_py3/lib/python3.6/http/client.py", line 297, in begin
    version, status, reason = self._read_status()
  File "//anaconda/envs/sb_py3/lib/python3.6/http/client.py", line 266, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "sb_automation.py", line 217, in <module>
    sb = pysb.SbSession(env=None).login(useremail, password)
  File "//anaconda/envs/sb_py3/lib/python3.6/site-packages/sciencebasepy/SbSession.py", line 105, in login
    self._session.post(self._josso_url, data=payload, headers=headers)
  File "//anaconda/envs/sb_py3/lib/python3.6/site-packages/requests/sessions.py", line 555, in post
    return self.request('POST', url, data=data, json=json, **kwargs)
  File "//anaconda/envs/sb_py3/lib/python3.6/site-packages/requests/sessions.py", line 508, in request
    resp = self.send(prep, **send_kwargs)
  File "//anaconda/envs/sb_py3/lib/python3.6/site-packages/requests/sessions.py", line 618, in send
    r = adapter.send(request, **kwargs)
  File "//anaconda/envs/sb_py3/lib/python3.6/site-packages/requests/adapters.py", line 490, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',))

Both times this has happened, it was while uploading to the pts_trans_ubw page. This time it happened on xml 49 and last time on xml 64. This time it printed "UPLOAD COMPLETED" before the error.

esturdivant-usgs commented 5 years ago

At least some of the time, the time out is happening when I attempt to log in.

Is it related to when I log in at the same time through another means? Don't think so because it happened after I left for the day yesterday.

esturdivant-usgs commented 5 years ago

I switched sb_automation to use my log_in() function instead of the try except log-in check I had been using. So far, I haven't encountered a time out error.

esturdivant-usgs commented 5 years ago

Maybe use ping() in sciencebasepy to check whether SB is available.