jjjake / internetarchive

A Python and Command-Line Interface to Archive.org
GNU Affero General Public License v3.0
1.61k stars 218 forks source link

MaxRetryError and SSLEOFError when uploading #616

Open serpentinesr opened 12 months ago

serpentinesr commented 12 months ago

Trying to upload couple of videos and getting this error ~1 minute into the upload:

requests.exceptions.SSLError: (MaxRetryError("HTTPSConnectionPool(host='s3.us.archive.org', port=443): Max retries exceeded with url: /f%20Evil%20%281973%29.mp4 (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:2426)')))"), 'https://s3.us.archive.org/f%20Evil%20%281973%29.mp4')

Anyone faced a similar issue?

hodgestar commented 10 months ago

I'm facing a similar issue. Does anyone know what might be causing it?

hodgestar commented 10 months ago

I regenerated my IAS API tokens and then the issue disappeared.

aeongdesu commented 7 months ago

I tried to regenerate IAS api token too, but it didn't fix as well..

mauforonda commented 7 months ago

I've found that some archive errors hide behind that SSL message. You could try disabling SSL and seeing what happens. In my case, this prompted a new error suggesting my identifier name was too long. Fixing that made things wholesome.

jjjake commented 7 months ago

@serpentinesr @aeongdesu can you provide the code that is returning this error?

In serpentinesr's error, the S3 URL does not look right to me. It appears to be missing an identifier (e.g. S3 bucket name). IA-S3 URLs should be formatted as http://s3.us.archive.org/{identifier}/{filename}.

Note that the whole file will be uploaded before an S3 error is returned. This is due to the lack of 100-continue support in Python unfortunately. That's why it appears to have uploaded before returning the error.

Shogun38 commented 5 months ago

I've been getting the same error since last night. I got some uploads in then the error popped up and now nothing will upload.

G:\Scans\Scanner Output>upload.bat
Weekly-reader-senior-edition-apr-10-1992:
 uploading Weekly Reader - Senior Edition - Vol. 46 Issue 24 - Apr 10 1992.cbz:  45%|▍| 28/urllib3.exceptions.SSLError: EOF occurred in violation of protocol (_ssl.c:2423)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\****\AppData\Local\Programs\Python\Python311\Lib\site-packages\requests\adapters.py", line 486, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "C:\Users\****\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connectionpool.py", line 844, in urlopen
    retries = retries.increment(
              ^^^^^^^^^^^^^^^^^^
  File "C:\Users\****\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\util\retry.py", line 515, in increment
    raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='s3.us.archive.org', port=443): Max retries exceeded with url: /Weekly-reader-senior-edition-apr-10-1992/Weekly%20Reader%20-%20Senior%20Edition%20-%20Vol.%2046%20Issue%2024%20-%20Apr%2010%201992.cbz (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:2423)')))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Users\****\AppData\Local\Programs\Python\Python311\Scripts\ia.exe\__main__.py", line 7, in <module>
  File "C:\Users\****\AppData\Local\Programs\Python\Python311\Lib\site-packages\internetarchive\cli\ia.py", line 175, in main
    sys.exit(ia_module.main(argv, session))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\****\AppData\Local\Programs\Python\Python311\Lib\site-packages\internetarchive\cli\ia_upload.py", line 310, in main
    r = _upload_files(item, local_file, upload_kwargs_copy, prev_identifier,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\****\AppData\Local\Programs\Python\Python311\Lib\site-packages\internetarchive\cli\ia_upload.py", line 101, in _upload_files
    response = item.upload(files, **upload_kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\****\AppData\Local\Programs\Python\Python311\Lib\site-packages\internetarchive\item.py", line 1277, in upload
    resp = self.upload_file(body,
           ^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\****\AppData\Local\Programs\Python\Python311\Lib\site-packages\internetarchive\item.py", line 1089, in upload_file
    response = self.session.send(prepared_request,
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\****\AppData\Local\Programs\Python\Python311\Lib\site-packages\internetarchive\session.py", line 540, in send
    raise e
  File "C:\Users\****\AppData\Local\Programs\Python\Python311\Lib\site-packages\internetarchive\session.py", line 537, in send
    reraise_modify(e, e.request.url, prepend=False)  # type: ignore
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\****\AppData\Local\Programs\Python\Python311\Lib\site-packages\internetarchive\session.py", line 534, in send
    r = super().send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\****\AppData\Local\Programs\Python\Python311\Lib\site-packages\requests\sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\****\AppData\Local\Programs\Python\Python311\Lib\site-packages\requests\adapters.py", line 517, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: (MaxRetryError("HTTPSConnectionPool(host='s3.us.archive.org', port=443): Max retries exceeded with url: /Weekly-reader-senior-edition-apr-10-1992/Weekly%20Reader%20-%20Senior%20Edition%20-%20Vol.%2046%20Issue%2024%20-%20Apr%2010%201992.cbz (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:2423)')))"), 'https://s3.us.archive.org/Weekly-reader-senior-edition-apr-10-1992/Weekly%20Reader%20-%20Senior%20Edition%20-%20Vol.%2046%20Issue%2024%20-%20Apr%2010%201992.cbz')
 uploading Weekly Reader - Senior Edition - Vol. 46 Issue 24 - Apr 10 1992.cbz:  45%|████▌     | 28/62 [01:12<01:28,  2.59s/MiB]
Press any key to continue . . .
Shogun38 commented 5 months ago

I attempted to run the upload.bat via --insecure as mentioned above and got the following error:

Microsoft Windows [Version 10.0.19044.1645]
(c) Microsoft Corporation. All rights reserved.

G:\Scans\Scanner Output>upload.bat
Weekly-reader-senior-edition-apr-10-1992:
 uploading Weekly Reader - Senior Edition - Vol. 46 Issue 24 - Apr 10 1992.cbz:  55%|▌| 34/Traceback (most recent call last):
  File "C:\Users\****\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connectionpool.py", line 790, in urlopen
    response = self._make_request(
               ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\****\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connectionpool.py", line 496, in _make_request
    conn.request(
  File "C:\Users\****\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connection.py", line 402, in request
    self.send(chunk)
  File "C:\Users\****\AppData\Local\Programs\Python\Python311\Lib\http\client.py", line 1002, in send
    self.sock.sendall(data)
ConnectionAbortedError: [WinError 10053] An established connection was aborted by the software in your host machine

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\****\AppData\Local\Programs\Python\Python311\Lib\site-packages\requests\adapters.py", line 486, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "C:\Users\****\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connectionpool.py", line 844, in urlopen
    retries = retries.increment(
              ^^^^^^^^^^^^^^^^^^
  File "C:\Users\****\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\util\retry.py", line 470, in increment
    raise reraise(type(error), error, _stacktrace)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\****\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\util\util.py", line 38, in reraise
    raise value.with_traceback(tb)
  File "C:\Users\****\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connectionpool.py", line 790, in urlopen
    response = self._make_request(
               ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\****\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connectionpool.py", line 496, in _make_request
    conn.request(
  File "C:\Users\****\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connection.py", line 402, in request
    self.send(chunk)
  File "C:\Users\****\AppData\Local\Programs\Python\Python311\Lib\http\client.py", line 1002, in send
    self.sock.sendall(data)
urllib3.exceptions.ProtocolError: ('Connection aborted.', ConnectionAbortedError(10053, 'An established connection was aborted by the software in your host machine', None, 10053, None))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Users\****\AppData\Local\Programs\Python\Python311\Scripts\ia.exe\__main__.py", line 7, in <module>
  File "C:\Users\****\AppData\Local\Programs\Python\Python311\Lib\site-packages\internetarchive\cli\ia.py", line 175, in main
    sys.exit(ia_module.main(argv, session))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\****\AppData\Local\Programs\Python\Python311\Lib\site-packages\internetarchive\cli\ia_upload.py", line 310, in main
    r = _upload_files(item, local_file, upload_kwargs_copy, prev_identifier,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\****\AppData\Local\Programs\Python\Python311\Lib\site-packages\internetarchive\cli\ia_upload.py", line 101, in _upload_files
    response = item.upload(files, **upload_kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\****\AppData\Local\Programs\Python\Python311\Lib\site-packages\internetarchive\item.py", line 1277, in upload
    resp = self.upload_file(body,
           ^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\****\AppData\Local\Programs\Python\Python311\Lib\site-packages\internetarchive\item.py", line 1089, in upload_file
    response = self.session.send(prepared_request,
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\****\AppData\Local\Programs\Python\Python311\Lib\site-packages\internetarchive\session.py", line 540, in send
    raise e
  File "C:\Users\****\AppData\Local\Programs\Python\Python311\Lib\site-packages\internetarchive\session.py", line 537, in send
    reraise_modify(e, e.request.url, prepend=False)  # type: ignore
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\****\AppData\Local\Programs\Python\Python311\Lib\site-packages\internetarchive\session.py", line 534, in send
    r = super().send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\****\AppData\Local\Programs\Python\Python311\Lib\site-packages\requests\sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\****\AppData\Local\Programs\Python\Python311\Lib\site-packages\requests\adapters.py", line 501, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: (ProtocolError('Connection aborted.', ConnectionAbortedError(10053, 'An established connection was aborted by the software in your host machine', None, 10053, None)), 'http://s3.us.archive.org/Weekly-reader-senior-edition-apr-10-1992/Weekly%20Reader%20-%20Senior%20Edition%20-%20Vol.%2046%20Issue%2024%20-%20Apr%2010%201992.cbz')
 uploading Weekly Reader - Senior Edition - Vol. 46 Issue 24 - Apr 10 1992.cbz:  55%|█████▍    | 34/62 [01:12<00:59,  2.14s/MiB]
Press any key to continue . . .