lisphilar / covid19-sir

CovsirPhy: Python library for COVID-19 analysis with phase-dependent SIR-derived ODE models.
https://lisphilar.github.io/covid19-sir/
Apache License 2.0
109 stars 43 forks source link

[Bug] URLError when downloading Japan data with DataDownloader #1321

Closed Venkatakrishnan-Ramesh closed 1 year ago

Venkatakrishnan-Ramesh commented 1 year ago

Checkbox

Summary

Error while Retrieving COVID-19 dataset in Japan from https://github.com/lisphilar/covid19-sir/data/japan and getting SSLError: [SSL] unknown error (_ssl.c:1131)

Reproducible example script

import covsirphy as cs
print(cs.__version__)

The current outputs

---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
File ~/miniconda3/envs/rapids-22.12/lib/python3.8/site-packages/covsirphy/cleaning/cbase.py:93, in CleaningBase.load(urlpath, header, columns, dtype)
     92 try:
---> 93     return dd.read_csv(urlpath, blocksize=None, **kwargs).compute()
     94 except (FileNotFoundError, UnicodeDecodeError, pd.errors.ParserError):

File ~/miniconda3/envs/rapids-22.12/lib/python3.8/site-packages/dask/dataframe/io/csv.py:700, in make_reader.<locals>.read(urlpath, blocksize, lineterminator, compression, sample, enforce, assume_missing, storage_options, include_path_column, **kwargs)
    688 def read(
    689     urlpath,
    690     blocksize="default",
   (...)
    698     **kwargs,
    699 ):
--> 700     return read_pandas(
    701         reader,
    702         urlpath,
    703         blocksize=blocksize,
    704         lineterminator=lineterminator,
    705         compression=compression,
    706         sample=sample,
    707         enforce=enforce,
    708         assume_missing=assume_missing,
    709         storage_options=storage_options,
    710         include_path_column=include_path_column,
    711         **kwargs,
    712     )

File ~/miniconda3/envs/rapids-22.12/lib/python3.8/site-packages/dask/dataframe/io/csv.py:535, in read_pandas(reader, urlpath, blocksize, lineterminator, compression, sample, enforce, assume_missing, storage_options, include_path_column, **kwargs)
    534 b_lineterminator = lineterminator.encode()
--> 535 b_out = read_bytes(
    536     urlpath,
    537     delimiter=b_lineterminator,
    538     blocksize=blocksize,
    539     sample=sample,
    540     compression=compression,
    541     include_path=include_path_column,
    542     **(storage_options or {}),
    543 )
    545 if include_path_column:

SSLError                                  Traceback (most recent call last)
File ~/miniconda3/envs/rapids-22.12/lib/python3.8/urllib/request.py:1354, in AbstractHTTPHandler.do_open(self, http_class, req, **http_conn_args)
   1353 try:
-> 1354     h.request(req.get_method(), req.selector, req.data, headers,
   1355               encode_chunked=req.has_header('Transfer-encoding'))
   1356 except OSError as err: # timeout error

File ~/miniconda3/envs/rapids-22.12/lib/python3.8/http/client.py:1256, in HTTPConnection.request(self, method, url, body, headers, encode_chunked)
   1255 """Send a complete request to the server."""
-> 1256 self._send_request(method, url, body, headers, encode_chunked)

File ~/miniconda3/envs/rapids-22.12/lib/python3.8/http/client.py:1302, in HTTPConnection._send_request(self, method, url, body, headers, encode_chunked)
   1301     body = _encode(body, 'body')
-> 1302 self.endheaders(body, encode_chunked=encode_chunked)

File ~/miniconda3/envs/rapids-22.12/lib/python3.8/http/client.py:1251, in HTTPConnection.endheaders(self, message_body, encode_chunked)
   1250     raise CannotSendHeader()
-> 1251 self._send_output(message_body, encode_chunked=encode_chunked)

File ~/miniconda3/envs/rapids-22.12/lib/python3.8/http/client.py:1011, in HTTPConnection._send_output(self, message_body, encode_chunked)
   1010 del self._buffer[:]
-> 1011 self.send(msg)
   1013 if message_body is not None:
   1014 
   1015     # create a consistent interface to message_body

File ~/miniconda3/envs/rapids-22.12/lib/python3.8/http/client.py:951, in HTTPConnection.send(self, data)
    950 if self.auto_open:
--> 951     self.connect()
    952 else:

File ~/miniconda3/envs/rapids-22.12/lib/python3.8/http/client.py:1425, in HTTPSConnection.connect(self)
   1423     server_hostname = self.host
-> 1425 self.sock = self._context.wrap_socket(self.sock,
   1426                                       server_hostname=server_hostname)

File ~/miniconda3/envs/rapids-22.12/lib/python3.8/ssl.py:500, in SSLContext.wrap_socket(self, sock, server_side, do_handshake_on_connect, suppress_ragged_eofs, server_hostname, session)
    494 def wrap_socket(self, sock, server_side=False,
    495                 do_handshake_on_connect=True,
    496                 suppress_ragged_eofs=True,
    497                 server_hostname=None, session=None):
    498     # SSLSocket class handles server_hostname encoding before it calls
    499     # ctx._wrap_socket()
--> 500     return self.sslsocket_class._create(
    501         sock=sock,
    502         server_side=server_side,
    503         do_handshake_on_connect=do_handshake_on_connect,
    504         suppress_ragged_eofs=suppress_ragged_eofs,
    505         server_hostname=server_hostname,
    506         context=self,
    507         session=session
    508     )

File ~/miniconda3/envs/rapids-22.12/lib/python3.8/ssl.py:1040, in SSLSocket._create(cls, sock, server_side, do_handshake_on_connect, suppress_ragged_eofs, server_hostname, context, session)
   1039             raise ValueError("do_handshake_on_connect should not be specified for non-blocking sockets")
-> 1040         self.do_handshake()
   1041 except (OSError, ValueError):

File ~/miniconda3/envs/rapids-22.12/lib/python3.8/ssl.py:1309, in SSLSocket.do_handshake(self, block)
   1308         self.settimeout(None)
-> 1309     self._sslobj.do_handshake()
   1310 finally:

SSLError: [SSL] unknown error (_ssl.c:1131)

Expected outputs

No response

Environment

- CovsirPhy version: 2.28.0
- Python version: 3.8.15

Package manager (required if installation issue)

pip

Platform (required if installation issue)

Ubuntu

Additional Context

No response

lisphilar commented 1 year ago

This may be related to #1319, an issue resolved at the latest development version (3.0.0.dev8). I will try to release patch for 2.28.x backporting.

I failed in reproducing the error with 2.28.0 and Google Colab (Python 3.8.16). Is the script of my notebook correct? https://gist.github.com/lisphilar/0413f34bf3f97e1d78141cd61cc43b8f