fatiando / pooch

A friend to fetch your data files
https://www.fatiando.org/pooch
Other
630 stars 76 forks source link

Download from URL with expired SSL certificate #360

Closed tomasstolker closed 1 year ago

tomasstolker commented 1 year ago

Thanks a lot for providing this really helpful tool!

I am trying to download a file from an URL that has an expired SSL certificate. I know that the URL is safe to use so I want to try to circumvent the error. Any suggestion on how to do that with pooch?

The error that I get is the following, which is caused by urllib3 / ssl:

_ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFYFAILED] certificate verify failed: certificate has expired

dokempf commented 1 year ago

Hey @tomasstolker! pooch is using the requests library under the hood which allows you to disable SSL checking. The way to pass such an option to pooch would be to manually instantiate the HTTPDownloader and give it the correct additional options:

import pooch

downloader = pooch.HTTPDownloader(verify=False)
filename = pooch.retrieve(URL, known_hash=HASH, downloader=downloader)

Note that I tested the above snippet, but only with a server that also has a certificate.

tomasstolker commented 1 year ago

Thanks a lot for your quick reply @dokempf! That works perfectly!

santisoler commented 1 year ago

Thanks @tomasstolker for opening this issue, and thanks @dokempf for the reply!

I think this issue has been resolves, so I'm closing it.

I still want to mention (specially for future readers) that disabling SSL verification is not a secure practice and it could potentially lead to getting malware in our systems.

🚨🚨🚨 So, BE EXTREMELY CAUTIOUS when using verify=False, and I highly discourage distributing code with that option. 🚨🚨🚨

leouieda commented 1 year ago

Just wanted to second everything that @santisoler said 😬