search

mroberge / hydrofunctions

A suite of convenience functions for working with hydrology data in an interactive Python session.
MIT License
66 stars 28 forks source link

SSL Certification error #108

Open mrranck opened 3 years ago

mrranck commented 3 years ago

Description

In one of my client environments, certain python libraries like siphon produce SSL certification errors. Siphon has a session manager that can either bypass this or add a certificate. Is there a similar kwarg or capability for hydrofunctions?

What I Did

Commands: current_test = hf.NWIS(['03352988'], 'iv', start_date=begin, end_date=today)

Traceback: requests.exceptions.SSLError: HTTPSConnectionPool(host='waterservices.usgs.gov', port=443): Max retries exceeded with url: /nwis/iv/?format=json%2C1.1&sites=03352988&startDT=2021-06-07&endDT=2021-07-08 (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1131)')))


Thank you!
mroberge commented 3 years ago

Hi @mrranck ! Thank you for filing this issue.

I tried : 

test = hf.NWIS(['03352988'], 'iv', start_date='2021-06-07', end_date='2021-07-08')

...and everything seemed to work as expected.

Looking at your error, it says that 'Max retries exceeded'... The NWIS does not want to be bombarded with the same request over and over, so I've created a 'file' parameter for hf.NWIS that you can use to store your data after your first request and then use the stored values after that. It is also possible to set up a caching system in the background so you don't spam the NWIS with your requests.

However, you got an error from requests, not an error message from the USGS NWIS. ???

As far as the certificate issue goes... I don't know anything about that! I searched online and came up with this: https://stackoverflow.com/a/12864892/2600766

I don't know anything about Siphon, but I looked it up and it looks interesting. I didn't design my interface for hydrofunctions to manage sessions- I organized it around individual requests for data.

I'm sorry that I don't have any answers for you!

If you can describe the problem in more detail or explain some of the issues you raised, perhaps I can help you with a work around?

mrranck commented 3 years ago

Thanks for the quick response! The reason the max retries were exceeded is that it kept trying again and again to access NWIS but failed to very certificate and and connect to the https at NWIS. What's in the stack overflow post would work:

requests.get('https://example.com', verify=False)

But I would need to know where to put that line.

mroberge commented 3 years ago

According to that stack overflow post, the "verify=False" solution would only be good for a one-time, temporary, quick hack. It's not a safe solution to use for an app or a software package.