NASA-IMPACT / pyQuARC

The pyQuARC tool reads and evaluates metadata records with a focus on the consistency and robustness of the metadata. pyQuARC flags opportunities to improve or add to contextual metadata information in order to help the user connect to relevant data products. pyQuARC also ensures that information common to both the data product and the file-level metadata are consistent and compatible. pyQuARC frees up human evaluators to make more sophisticated assessments such as whether an abstract accurately describes the data and provides the correct contextual information. The base pyQuARC package assesses descriptive metadata used to catalog Earth observation data products and files. As open source software, pyQuARC can be adapted and customized by data providers to allow for quality checks that evolve with their needs, including checking metadata not covered in base package.
Apache License 2.0
19 stars 0 forks source link

Requesting Python 3.10 Upgrade for QuARC Lambda Environment Due to TLS 1.3 Support #280

Open rajeshpandey2053 opened 3 months ago

rajeshpandey2053 commented 3 months ago

Describe the bug QuARC currently relies on Lambda for deployment, utilizing Python 3.8 as the runtime environment. However, pyQuARC dependencies are also tied to Python 3.8.

We've encountered an issue stemming from the older version of OpenSSL installed in the Python 3.8 runtime environment, which results in errors when making requests to certain web endpoints.

To mitigate this, we propose upgrading the Python runtime environment to version 3.10.

To Reproduce Steps to reproduce the behavior:

  1. Go to AWS lambda and set the runtime as Python 3.8
  2. Run the following code in the handler.py
    import requests
    url = "https://www.atmosp.physics.utoronto.ca/MOPITT/home.html"
    response=requests.get(url, headers=headers, timeout=10).status_code
    print(response)
  3. See the error as follows { "errorMessage": "HTTPSConnectionPool(host='www.atmosp.physics.utoronto.ca', port=443): Max retries exceeded with url: /MOPITT/home.html (Caused by SSLError(SSLError(1, '[SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:1131)')))", "errorType": "SSLError", "stackTrace": [ " File \"/var/task/lambda_function.py\", line 132, in lambda_handler\n \"body\": json.dumps(requests.get(url).text),\n", " File \"/var/task/requests/api.py\", line 76, in get\n return request('get', url, params=params, **kwargs)\n", " File \"/var/task/requests/api.py\", line 61, in request\n return session.request(method=method, url=url, **kwargs)\n", " File \"/var/task/requests/sessions.py\", line 530, in request\n resp = self.send(prep, **send_kwargs)\n", " File \"/var/task/requests/sessions.py\", line 643, in send\n r = adapter.send(request, **kwargs)\n", " File \"/var/task/requests/adapters.py\", line 514, in send\n raise SSLError(e, request=request)\n" ] }

Expected behavior Upgrading the Python runtime environment to 3.10 should align the dependencies across QuARC and Lambda, resolving the OpenSSL version discrepancy and ensuring smooth operation when making web requests.

Additional context

rajeshpandey2053 commented 3 months ago

Upgraded python to version 3.9 in QuARC deployment. Link to PR