sfu-db / dataprep

Open-source low code data preparation library in python. Collect, clean and visualization your data in python with a few lines of code.
http://dataprep.ai
MIT License
2.01k stars 204 forks source link

SSLCertVerificationError with dataprep.connector but not requests module? #788

Closed brendanartley closed 2 years ago

brendanartley commented 2 years ago

Describe the bug

I am trying to use the dataprep.connector with the NHL API that does not required API Keys.

NHL_API Documentation

I am get a response with no errors when I use the requests module like in the following code block.

import requests

r = requests.get("https://records.nhl.com/site/api/attendance")
r.json()

Yet when I try and do the same with the dataprep.connector, I am getting a SSLCertVerificationError and a ClientConnectorCertificateError.

conn = Connector(config_path="./config")
attendance = await conn.query("attendance")
SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1108)

ClientConnectorCertificateError: Cannot connect to host records.nhl.com:443 ssl:True [SSLCertVerificationError: (1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1108)')]

My config file is very simple and looks like the following.

{
    "version": 1,
    "request": {
        "url": "http://records.nhl.com/site/api/attendance",
        "method": "GET",
        "params": {
            "cayenneExp": false
        }
    },
    "response": {
        "ctype": "application/json",
        "tablePath": "$.data[*]",
        "schema": {
            "Id": {
                "target": "$.id",
                "type": "int"
            },
            "seasonId": {
                "target": "$.seasonId",
                "type": "int"
            },
            "playoffAttendance": {
                "target": "$.playoffAttendance",
                "type": "int"
            },
            "regularAttendance": {
                "target": "$.regularAttendance",
                "type": "int"
            },
            "totalAttendance": {
                "target": "$.totalAttendance",
                "type": "int"
            }
        },
        "orient": "records"
    }
}

I dont see why this would work without errors with the requests module and not with the dataprep.connector? If anyone knows why this was the case let me know!

Desktop (please complete the following information):

Additional context

I have tried the following to fix the error so far.

  1. Making sure I am using the most up to date certifi version pip install certifi --upgrade

  2. Attempting the API call with both http and https

dovahcrow commented 2 years ago

Connector is using aiohttp instead of requests so I think this might be the issue although I cannot reproduce the issue on my side..

dovahcrow commented 2 years ago

Are you using Windows? It looks like the issue might have a similar cause to this one https://github.com/Rapptz/discord.py/issues/4159.

brendanartley commented 2 years ago

@Rapptz's solution worked. Thank you!