search

LowellObservatory / NightShift

Collection of sub-modules used in the NightWatch (and other Night*) project
Mozilla Public License 2.0
1 stars 1 forks source link

Unhandled connection error #21

Open astrobokonon opened 3 years ago

astrobokonon commented 3 years ago

A rare crash! First I've seen of this one

2021-08-26 18:10:56,479 INFO     already downloaded!
2021-08-26 18:10:56,479 INFO     ./outputs/goes//raws//20212381758554_C13.nc
2021-08-26 18:10:56,479 INFO     already downloaded!
2021-08-26 18:10:56,480 INFO     Querying:
2021-08-26 18:10:56,480 INFO     ABI-L2-CMIPC/2021/238/18/
2021-08-26 18:10:56,819 INFO     ./outputs/goes//raws//20212381803555_C13.nc
2021-08-26 18:10:56,819 INFO     already downloaded!
2021-08-26 18:11:27,402 ERROR    Traceback (most recent call last):
2021-08-26 18:11:27,403 ERROR      File "/opt/conda/lib/python3.8/site-packages/urllib3/response.py", line 436, in _error_catcher
2021-08-26 18:11:27,403 ERROR    yield
2021-08-26 18:11:27,403 ERROR      File "/opt/conda/lib/python3.8/site-packages/urllib3/response.py", line 518, in read
2021-08-26 18:11:27,403 ERROR    data = self._fp.read(amt) if not fp_closed else b""
2021-08-26 18:11:27,403 ERROR      File "/opt/conda/lib/python3.8/http/client.py", line 458, in read
2021-08-26 18:11:27,404 ERROR    n = self.readinto(b)
2021-08-26 18:11:27,404 ERROR      File "/opt/conda/lib/python3.8/http/client.py", line 502, in readinto
2021-08-26 18:11:27,404 ERROR    n = self.fp.readinto(b)
2021-08-26 18:11:27,404 ERROR      File "/opt/conda/lib/python3.8/socket.py", line 669, in readinto
2021-08-26 18:11:27,404 ERROR    return self._sock.recv_into(b)
2021-08-26 18:11:27,404 ERROR      File "/opt/conda/lib/python3.8/ssl.py", line 1241, in recv_into
2021-08-26 18:11:27,416 ERROR    return self.read(nbytes, buffer)
2021-08-26 18:11:27,417 ERROR      File "/opt/conda/lib/python3.8/ssl.py", line 1099, in read
2021-08-26 18:11:27,417 ERROR    return self._sslobj.read(len, buffer)
2021-08-26 18:11:27,417 ERROR    ConnectionResetError
2021-08-26 18:11:27,417 ERROR    :
2021-08-26 18:11:27,417 ERROR    [Errno 104] Connection reset by peer
2021-08-26 18:11:27,417 ERROR
During handling of the above exception, another exception occurred:
2021-08-26 18:11:27,417 ERROR    Traceback (most recent call last):
2021-08-26 18:11:27,417 ERROR      File "looper_goes16aws.py", line 185, in <module>
2021-08-26 18:11:27,431 ERROR    main(outdir, creds, sleep=90.,
2021-08-26 18:11:27,431 ERROR      File "looper_goes16aws.py", line 107, in main
2021-08-26 18:11:27,431 ERROR    ffiles = aws.GOESAWSgrab(aws_keyid, aws_secretkey, when, dout,
2021-08-26 18:11:27,431 ERROR      File "/home/lig/Codes/NightShift/nightshift/goes/aws.py", line 124, in GOESAWSgrab
2021-08-26 18:11:27,436 ERROR    com.aws.downloadFromS3(buck, objs, oname)
2021-08-26 18:11:27,436 ERROR      File "/home/lig/Codes/NightShift/nightshift/common/aws.py", line 43, in downloadFromS3
2021-08-26 18:11:27,437 ERROR    buck.download_file(objs.key, oname)
2021-08-26 18:11:27,437 ERROR      File "/opt/conda/lib/python3.8/site-packages/boto3/s3/inject.py", line 244, in bucket_download_file
2021-08-26 18:11:27,445 ERROR    return self.meta.client.download_file(
2021-08-26 18:11:27,445 ERROR      File "/opt/conda/lib/python3.8/site-packages/boto3/s3/inject.py", line 170, in download_file
2021-08-26 18:11:27,445 ERROR    return transfer.download_file(
2021-08-26 18:11:27,445 ERROR      File "/opt/conda/lib/python3.8/site-packages/boto3/s3/transfer.py", line 307, in download_file
2021-08-26 18:11:27,449 ERROR    future.result()
2021-08-26 18:11:27,449 ERROR      File "/opt/conda/lib/python3.8/site-packages/s3transfer/futures.py", line 106, in result
2021-08-26 18:11:27,449 ERROR    return self._coordinator.result()
2021-08-26 18:11:27,449 ERROR      File "/opt/conda/lib/python3.8/site-packages/s3transfer/futures.py", line 265, in result
2021-08-26 18:11:27,456 ERROR    raise self._exception
2021-08-26 18:11:27,456 ERROR      File "/opt/conda/lib/python3.8/site-packages/s3transfer/tasks.py", line 126, in __call__
2021-08-26 18:11:27,456 ERROR    return self._execute_main(kwargs)
2021-08-26 18:11:27,456 ERROR      File "/opt/conda/lib/python3.8/site-packages/s3transfer/tasks.py", line 150, in _execute_main
2021-08-26 18:11:27,457 ERROR    return_value = self._main(**kwargs)
2021-08-26 18:11:27,457 ERROR      File "/opt/conda/lib/python3.8/site-packages/s3transfer/download.py", line 521, in _main
2021-08-26 18:11:27,457 ERROR    for chunk in chunks:
2021-08-26 18:11:27,457 ERROR      File "/opt/conda/lib/python3.8/site-packages/s3transfer/download.py", line 649, in __next__
2021-08-26 18:11:27,463 ERROR    chunk = self._body.read(self._chunksize)
2021-08-26 18:11:27,463 ERROR      File "/opt/conda/lib/python3.8/site-packages/s3transfer/utils.py", line 570, in read
2021-08-26 18:11:27,475 ERROR    value = self._stream.read(*args, **kwargs)
2021-08-26 18:11:27,475 ERROR      File "/opt/conda/lib/python3.8/site-packages/botocore/response.py", line 77, in read
2021-08-26 18:11:27,479 ERROR    chunk = self._raw_stream.read(amt)
2021-08-26 18:11:27,479 ERROR      File "/opt/conda/lib/python3.8/site-packages/urllib3/response.py", line 540, in read
2021-08-26 18:11:27,479 ERROR    raise IncompleteRead(self._fp_bytes_read, self.length_remaining)
2021-08-26 18:11:27,480 ERROR      File "/opt/conda/lib/python3.8/contextlib.py", line 131, in __exit__
2021-08-26 18:11:27,497 ERROR    self.gen.throw(type, value, traceback)
2021-08-26 18:11:27,497 ERROR      File "/opt/conda/lib/python3.8/site-packages/urllib3/response.py", line 454, in _error_catcher
2021-08-26 18:11:27,497 ERROR    raise ProtocolError("Connection broken: %r" % e, e)
2021-08-26 18:11:27,497 ERROR    urllib3.exceptions
2021-08-26 18:11:27,497 ERROR    .
2021-08-26 18:11:27,497 ERROR    ProtocolError
2021-08-26 18:11:27,497 ERROR    :
2021-08-26 18:11:27,497 ERROR    ("Connection broken: ConnectionResetError(104, 'Connection reset by peer')", ConnectionResetError(104, 'Connection reset by peer'))
astrobokonon commented 3 years ago

Seems like it was a garden variety connection interruption, my guess is the site internet blipped briefly at an unlucky time. Needs a double check though.