search

indygreg / python-build-standalone

Produce redistributable builds of Python
BSD 3-Clause "New" or "Revised" License
1.71k stars 107 forks source link

Python unable to see CA certificates in Fedora/Rocky/Alma/RHEL systems #259

Open nonell-viavi opened 1 month ago

nonell-viavi commented 1 month ago

Hi,

I think there is an issue when attempting to use this precompiled binaries on a rhel-based system. I've read the quirks section, and I haven't seen a reference to this problem, that is why I'm raising an issue.

The reason we care about this funcionality is because we're trying to use this python + poetry to get "portable" dev environments accross a bunch of developers (using mise).

Issue description

The following code reproduces the problem on a rhel-based system:

docker run -it --rm rockylinux:8 bash -c \
  "dnf update -y && dnf install -y wget ca-certificates && \
  wget https://github.com/indygreg/python-build-standalone/releases/download/20240224/cpython-3.10.13+20240224-x86_64-unknown-linux-gnu-install_only.tar.gz && \
  tar -xzf cpython-3.10.13+20240224-x86_64-unknown-linux-gnu-install_only.tar.gz && \
  ./python/bin/python -c \"import urllib.request; f = urllib.request.urlopen('https://www.python.org'); print(f.read(100))\""
Logs Connecting to objects.githubusercontent.com (objects.githubusercontent.com)|185.199.108.133|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 27409290 (26M) [application/octet-stream] Saving to: 'cpython-3.10.13+20240224-x86_64-unknown-linux-gnu-install_only.tar.gz' cpython-3.10.13+20240224-x86_64-unknown-linux-gnu-in 100%[=====================================================================================================================>] 26.14M 8.59MB/s in 3.0s 2024-05-06 08:57:30 (8.59 MB/s) - 'cpython-3.10.13+20240224-x86_64-unknown-linux-gnu-install_only.tar.gz' saved [27409290/27409290] Traceback (most recent call last): File "/python/lib/python3.10/urllib/request.py", line 1348, in do_open h.request(req.get_method(), req.selector, req.data, headers, File "/python/lib/python3.10/http/client.py", line 1283, in request self._send_request(method, url, body, headers, encode_chunked) File "/python/lib/python3.10/http/client.py", line 1329, in _send_request self.endheaders(body, encode_chunked=encode_chunked) File "/python/lib/python3.10/http/client.py", line 1278, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "/python/lib/python3.10/http/client.py", line 1038, in _send_output self.send(msg) File "/python/lib/python3.10/http/client.py", line 976, in send self.connect() File "/python/lib/python3.10/http/client.py", line 1455, in connect self.sock = self._context.wrap_socket(self.sock, File "/python/lib/python3.10/ssl.py", line 513, in wrap_socket return self.sslsocket_class._create( File "/python/lib/python3.10/ssl.py", line 1104, in _create self.do_handshake() File "/python/lib/python3.10/ssl.py", line 1375, in do_handshake self._sslobj.do_handshake() ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1007) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "", line 1, in File "/python/lib/python3.10/urllib/request.py", line 216, in urlopen return opener.open(url, data, timeout) File "/python/lib/python3.10/urllib/request.py", line 519, in open response = self._open(req, data) File "/python/lib/python3.10/urllib/request.py", line 536, in _open result = self._call_chain(self.handle_open, protocol, protocol + File "/python/lib/python3.10/urllib/request.py", line 496, in _call_chain result = func(*args) File "/python/lib/python3.10/urllib/request.py", line 1391, in https_open return self.do_open(http.client.HTTPSConnection, req, File "/python/lib/python3.10/urllib/request.py", line 1351, in do_open raise URLError(err) urllib.error.URLError:

But the same python program, urlopen works just fine on a debian-based system.

docker run -it --rm debian:12 bash -c \
  "apt update -y && apt install -y wget ca-certificates && \
  wget https://github.com/indygreg/python-build-standalone/releases/download/20240224/cpython-3.10.13+20240224-x86_64-unknown-linux-gnu-install_only.tar.gz && \
  tar -xzf cpython-3.10.13+20240224-x86_64-unknown-linux-gnu-install_only.tar.gz && \
  ./python/bin/python -c \"import urllib.request; f = urllib.request.urlopen('https://www.python.org'); print(f.read(100))\""
Logs ... Connecting to objects.githubusercontent.com (objects.githubusercontent.com)|185.199.110.133|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 27409290 (26M) [application/octet-stream] Saving to: 'cpython-3.10.13+20240224-x86_64-unknown-linux-gnu-install_only.tar.gz' cpython-3.10.13+20240224-x86_64-unknown-linux-gnu-in 100%[=====================================================================================================================>] 26.14M 8.63MB/s in 3.0s 2024-05-06 08:53:54 (8.63 MB/s) - 'cpython-3.10.13+20240224-x86_64-unknown-linux-gnu-install_only.tar.gz' saved [27409290/27409290] b'\x1f\x8b\x08\x00\x00\x00\x00\x00\x00\x03\xed}\xebr\x1bG\xb2\xe6\x7fE\xcc;\x94\xa1X\x91\x1a\xab\x01\x82\xe0M\x14\t\x8f$S2=\xba\xd0\xa6d\x9d\xb3>\x0eE\xa3\xd1\x00\x9a\x04\xba\xa1\xbe\x90\x84g\xe6\x01\xce\x9f}\x85}\xc5}\x84\xfd2\xab\xaa\xbb\xfa\n@\xe4\xe8\xd8\x11\x9a\t\x8b@\xa3\xba.Yy\xaf\xcc\xac\xa3o\x86\x81\x13/\xe6'

Other things I've tried

I've tried installing pip-system-certs and certifi, using the provided pip, but neither of them fixed the issue.

Thank you for reading this far, and let me know if there is something else I can do to help. I'm stuck and unable to fix this by myself (I think).

dirkpetersen commented 1 month ago

I ran into the exact same thing on Python 3.10 (3.8 worked fine) , and used monkey patching in the sitecustomize.py to enable urllib ssl globally per ChatGPT recommendation: 

vi $(python -c "import site; print(site.getsitepackages()[0])")/sitecustomize.py

paste this

import ssl
try:
    import certifi
    # Define a function that returns a default SSL context with certifi's CA bundle
    def create_certifi_context(purpose=ssl.Purpose.SERVER_AUTH, *, cafile=None, capath=None, cadata=None):
        return ssl.create_default_context(purpose, cafile=certifi.where(), capath=capath, cadata=cadata)
    # Set the default SSL context creation function to use certifi's CA bundle
    ssl._create_default_https_context = create_certifi_context
except:
    print('certifi package not installed')