Respect REQUESTS_CA_BUNDLE as a provider of cert bundle at runtime.

[aalekhpatel07]

Fetching contents from a site with a custom SSL cert setup causes the following error:

SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer 
certificate (_ssl.c:1129)

Ideally, we'd like to respect custom cert bundles at runtime, and requests takes a REQUESTS_CA_BUNDLE approach for it. We could use the same here.

In the new version, the usage would look like this:

REQUESTS_CA_BUNDLE=/path/to/bundle frogmouth https://site-with-custom-ssl-setup/-/blob/main/README.md

[davep]

Just looking at the background to this and how this relates to the fact that we use httpx, not requests; is this relevant:

Requests supports REQUESTS_CA_BUNDLE which points to either a file or a directory. HTTPX supports the SSL_CERT_FILE (for a file) and SSL_CERT_DIR (for a directory) OpenSSL variables instead.

[aalekhpatel07]

That looks like a decent alternative. Thank you. I wonder if it is still worth to support REQUESTS_CA_BUNDLE even if just for the sake of completeness and fixing this leaky abstraction.

[davep]

I wonder if it is still worth to support REQUESTS_CA_BUNDLE even if just for the sake of completeness

Given that that seems specific to one particular library for one particular language, whereas the ones that httpx support seem very general on a system, I don't think I'm seeing the benefit really, unless I'm missing something? Surely if an environment needs to be doing this it would potentially need to be doing it for many applications, OpenSSL is pretty common, and so a user of frogmouth wouldn't even need to know the language used to write it, nor the library used to perform requests, to fix the problem.

Am I missing something here?

[aalekhpatel07]

seems reasonable. thanks for clarifying!