search

benedekrozemberczki / pytorch_geometric_temporal

PyTorch Geometric Temporal: Spatiotemporal Signal Processing with Neural Machine Learning Models (CIKM 2021)
MIT License
2.58k stars 367 forks source link

🐛 Fix SSL: CERTIFICATE_VERIFY_FAILED error #262

Open nithinmanoj10 opened 6 months ago

nithinmanoj10 commented 6 months ago

tl;dr

Made the fix for SSL: CERTIFICATE_VERIFY_FAILED error that is encountered while downloading the dataset for each dataloader. This is fixed by passing an unverified SSL context to the urllib.request.urlopen() method.

This pull request is in regards to the following issues #255, #243 and #236. I had also faced a similar issue when trying to download the datasets for the dataloader.

Reproducing the error

Ran the following Python script to load the METRLADatasetLoader 

import torch_geometric_temporal
from torch_geometric_temporal.dataset import METRLADatasetLoader

metrla = METRLADatasetLoader()

And encountered the following error

Traceback (most recent call last):
  File "/home/nithin/anaconda3/envs/seastar-new/lib/python3.10/urllib/request.py", line 1348, in do_open
    h.request(req.get_method(), req.selector, req.data, headers,
  File "/home/nithin/anaconda3/envs/seastar-new/lib/python3.10/http/client.py", line 1282, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/home/nithin/anaconda3/envs/seastar-new/lib/python3.10/http/client.py", line 1328, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/home/nithin/anaconda3/envs/seastar-new/lib/python3.10/http/client.py", line 1277, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/home/nithin/anaconda3/envs/seastar-new/lib/python3.10/http/client.py", line 1037, in _send_output
    self.send(msg)
  File "/home/nithin/anaconda3/envs/seastar-new/lib/python3.10/http/client.py", line 975, in send
    self.connect()
  File "/home/nithin/anaconda3/envs/seastar-new/lib/python3.10/http/client.py", line 1454, in connect
    self.sock = self._context.wrap_socket(self.sock,
  File "/home/nithin/anaconda3/envs/seastar-new/lib/python3.10/ssl.py", line 513, in wrap_socket
    return self.sslsocket_class._create(
  File "/home/nithin/anaconda3/envs/seastar-new/lib/python3.10/ssl.py", line 1071, in _create
    self.do_handshake()
  File "/home/nithin/anaconda3/envs/seastar-new/lib/python3.10/ssl.py", line 1342, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:997)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/mnt/c/Nithin stuff/Open Source Contributions/pytorch_geometric_temporal/torch_geometric_temporal/dataset/metr_la.py", line 25, in __init__
    self._read_web_data()
  File "/mnt/c/Nithin stuff/Open Source Contributions/pytorch_geometric_temporal/torch_geometric_temporal/dataset/metr_la.py", line 41, in _read_web_data
    self._download_url(url, os.path.join(self.raw_data_dir, "METR-LA.zip"))
  File "/mnt/c/Nithin stuff/Open Source Contributions/pytorch_geometric_temporal/torch_geometric_temporal/dataset/metr_la.py", line 28, in _download_url
    with urllib.request.urlopen(url) as dl_file:
  File "/home/nithin/anaconda3/envs/seastar-new/lib/python3.10/urllib/request.py", line 216, in urlopen
    return opener.open(url, data, timeout)
  File "/home/nithin/anaconda3/envs/seastar-new/lib/python3.10/urllib/request.py", line 519, in open
    response = self._open(req, data)
  File "/home/nithin/anaconda3/envs/seastar-new/lib/python3.10/urllib/request.py", line 536, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
  File "/home/nithin/anaconda3/envs/seastar-new/lib/python3.10/urllib/request.py", line 496, in _call_chain
    result = func(*args)
  File "/home/nithin/anaconda3/envs/seastar-new/lib/python3.10/urllib/request.py", line 1391, in https_open
    return self.do_open(http.client.HTTPSConnection, req,
  File "/home/nithin/anaconda3/envs/seastar-new/lib/python3.10/urllib/request.py", line 1351, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:997)>

And noticed that the tests for the dataset loaders were failing due to the similar error

pygt-pr-test

The fix for the error

The fix was inspired by what PyTorch Geometric had done to bypass the SSL: CERTIFICATE_VERIFY_FAILED error. All they did was pass an unverified SSL context to the urllib.request.urlopen() method call.

context = ssl._create_unverified_context()
self._dataset = json.loads(
    urllib.request.urlopen(url, context=context).read()
)

The PyTorch Geometric code were they did that to download the datasets can be found here.

A similar solution was also mentioned in a comment inside issue #236.

And upon running the tests, we are not encountering the previous errors anymore

pygt-pr-test-fix