EPA AQS URLs causing error

zmoon commented 1 year ago

EPA AQS URLs erroring in link check apparently due to their server not supporting OpenSSL 3 client
- e.g.
  (tutorial/aqs_pams: line 27) broken https://aqs.epa.gov/aqsweb/airdata/download_files.html - HTTPSConnectionPool(host='aqs.epa.gov', port=443): Max retries exceeded with url: /aqsweb/airdata/download_files.html (Caused by SSLError(SSLError(1, '[SSL: UNSAFE_LEGACY_RENEGOTIATION_DISABLED] unsafe legacy renegotiation disabled (_ssl.c:1129)')))
- seems this is affecting the AQS test as well (232e0e4): FAILED tests/test_aqs.py::test_aqs - requests.exceptions.SSLError: HTTPSConnectionPool(host='aqs.epa.gov', port=443): Max retries exceeded with url: /aqsweb/airdata/daily_PM10SPEC_2019.zip (Caused by SSLError(SSLError(1, '[SSL: UNSAFE_LEGACY_RENEGOTIATION_DISABLED] unsafe legacy renegotiation disabled (_ssl.c:1131)')))

zmoon commented 1 year ago

Some more info here

zmoon commented 1 year ago

Pinning openssl is an option I suppose (1.1.1q is the current < 3 release)

zmoon commented 1 year ago

Also for requests.get could try this method

bbakernoaa commented 1 year ago

@zmoon is this finished?

zmoon commented 1 year ago

Still working around it in CI. AFAIK as I know the situation hasn't changed. Did you have any luck contacting them about it?

zmoon commented 1 year ago

Just tested this locally, aqs.epa.gov URLs still have the same issue in link check and the test when using OpenSSL v3.

zmoon commented 9 months ago

In CI run for #125 today, the AQS test didn't fail with OpenSSL 3.1.2.

bbakernoaa commented 9 months ago

That’s great!

zmoon commented 9 months ago

Phil Dickerson from EPA says the server is officially fixed as of last week.

Will keep this issue open until PR removing the related extra CI cases etc.

noaa-oar-arl / monetio