Open hoel-bagard opened 3 months ago
ichard26
commented
3 months ago Could you try passing --use-deprecated=legacy-certs to the pip invocation? If that works, then we can narrow down the issue to the truststore feature (which seems likely given that pip 24.1 works which has truststore disabled by default). cc @sethmlarson
sethmlarson
commented
3 months ago I wasn't able to get the ValueError you're running into to reproduce, is there any other flag you're passing to pip or environment variable that would configure/disable certificate verification?
I tried using TestPyPI as an example "index-url", maybe it only triggers for other indices?
$ python -m pip --version
pip 24.2 from /tmp/pip-12906/venv/lib/python3.12/site-packages/pip (python 3.12)
$ python -m pip install urllib3 --index-url=https://test.pypi.org/simple --no-cache-dir
(succeeds)
hoel-bagard
commented
3 months ago Could you try passing --use-deprecated=legacy-certs to the pip invocation? If that works, then we can narrow down the issue to the truststore feature (which seems likely given that pip 24.1 works which has truststore disabled by default).
If I use --use-deprecated=legacy-certs, then the behavior becomes the same as with pip 24.1. It works if I give the pass to the CA using --cert (I guess that's because I haven't configured things properly, although I don't know what I'm missing).
I wasn't able to get the ValueError you're running into to reproduce, is there any other flag you're passing to pip or environment variable that would configure/disable certificate verification?
Not that I'm aware of. I saw in this issue that it got linked to proxy settings. I am behind a proxy, however I am unsetting all the proxy environment variable when running the pip commands (since the private index is in the local network).
I tried using TestPyPI as an example "index-url", maybe it only triggers for other indices?
I tried your command, and it worked without issue. I'm not sure how relevant it is, but I set the proxy environment variables when doing it. I'll try to replicate the issue in a simplified environment.
ichard26
commented
2 months ago (I guess that's because I haven't configured things properly, although I don't know what I'm missing).
Well, before pip 24.2, pip would only verify certificates against its vendored certificate bundle. That's what certifi is: it's a Python repackaging of Mozilla's certificate bundle. For a while, pip gained the ability to integrate with truststore which uses the system certificate store. However, this feature was disabled by default and had to be enabled manually until pip 24.2. In theory, "it should just work" if your private CA is configured properly at the system level with truststore enabled, but evidently somewhere in the chain (:P) it's not working...
A simplified or MRE would be greatly appreciated.
hoel-bagard
commented
2 months ago @ichard26 I've been able to reproduce the error using this repo.
I ran it at home on my personal computer, so no proxy was involved.
ichard26
commented
2 months ago @sethmlarson while I have your attention, perhaps you could look at this too? :)
There is an attached reproduction repository, but I was able to reproduce the error by simply pointing pip to a 127.0.0.1 HTTPS index with a custom CA/cert. I wasn't able to figure out what exactly goes wrong, but it seems like with truststore enabled, the check_hostname session attribute remains true despite urllib3 attempting to disable it.
ichard26
commented
1 week ago Heya, any updates @sethmlarson? It's too late for the 24.3 cycle, but I'd love to make truststore rock solid on supported platforms in pip 25.0.
sethmlarson
commented
1 week ago I haven't dug into this issue recently, but the latest release of truststore did make some changes to check_hostname, might be worth checking again.
rob4226
commented
1 week ago I'm seeing this error also with a self-signed root CA and private PyPi registry. Usually setting PIP_CERT works, but not with v24.2 and v24.3. I keep getting a ValueError: check_hostname requires server_hostname error.
File "/server-082/workspace/project-ea52f8/venv/lib/python3.12/site-packages/pip/_vendor/truststore/_api.py", line 96, in wrap_socket
ssl_sock = self._ctx.wrap_socket(
^^^^^^^^^^^^^^^^^^^^^^
File "/opt/tools/Python/3.12.7/x64/lib/python3.12/ssl.py", line 455, in wrap_socket
return self.sslsocket_class._create(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/tools/Python/3.12.7/x64/lib/python3.12/ssl.py", line 969, in _create
raise ValueError("check_hostname requires server_hostname")
ValueError: check_hostname requires server_hostname
Description
Attempting to install a package from a private package index with pip 24.2 results in a
ValueError: check_hostname requires server_hostnameerror.I am using the command below:
Expected behavior
Package is downloaded from the index without an error occuring.
pip version
24.2
Python version
3.12
OS
Linux
How to Reproduce
pip install pip==24.2pip install pip==24.1 --index-url=https://163.219.218.169/root/pypi --no-cache-dirValueError: check_hostname requires server_hostnameexception occurs.Output
Full traceback
``` WARNING: There was an error checking the latest version of pip. ERROR: Exception: Traceback (most recent call last): File "/home/hoel/work-hitachi/pypi-repository/.venv/lib/python3.12/site-packages/pip/_internal/cli/base_command.py", line 105, in _run_wrapper status = _inner_run() ^^^^^^^^^^^^ File "/home/hoel/work-hitachi/pypi-repository/.venv/lib/python3.12/site-packages/pip/_internal/cli/base_command.py", line 96, in _inner_run return self.run(options, args) ^^^^^^^^^^^^^^^^^^^^^^^ File "/home/hoel/work-hitachi/pypi-repository/.venv/lib/python3.12/site-packages/pip/_internal/cli/req_command.py", line 67, in wrapper return func(self, options, args) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/hoel/work-hitachi/pypi-repository/.venv/lib/python3.12/site-packages/pip/_internal/commands/install.py", line 379, in run requirement_set = resolver.resolve( ^^^^^^^^^^^^^^^^^ File "/home/hoel/work-hitachi/pypi-repository/.venv/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 95, in resolve result = self._result = resolver.resolve( ^^^^^^^^^^^^^^^^^ File "/home/hoel/work-hitachi/pypi-repository/.venv/lib/python3.12/site-packages/pip/_vendor/resolvelib/resolvers.py", line 546, in resolve state = resolution.resolve(requirements, max_rounds=max_rounds) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/hoel/work-hitachi/pypi-repository/.venv/lib/python3.12/site-packages/pip/_vendor/resolvelib/resolvers.py", line 397, in resolve self._add_to_criteria(self.state.criteria, r, parent=None) File "/home/hoel/work-hitachi/pypi-repository/.venv/lib/python3.12/site-packages/pip/_vendor/resolvelib/resolvers.py", line 173, in _add_to_criteria if not criterion.candidates: ^^^^^^^^^^^^^^^^^^^^ File "/home/hoel/work-hitachi/pypi-repository/.venv/lib/python3.12/site-packages/pip/_vendor/resolvelib/structs.py", line 156, in __bool__ return bool(self._sequence) ^^^^^^^^^^^^^^^^^^^^ File "/home/hoel/work-hitachi/pypi-repository/.venv/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 174, in __bool__ return any(self) ^^^^^^^^^ File "/home/hoel/work-hitachi/pypi-repository/.venv/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 162, inOther
I am still in the process of setting up the https part of my private package index, and while I was able to use it to some extend with
pip==24.1, I had to manually point to the CA certificate. So I cannot exclude the possibility of a wrong setup on my part.Code of Conduct