astral-sh / rye

a Hassle-Free Python Experience
https://rye.astral.sh
MIT License
13.6k stars 466 forks source link

Invalid peer certificate: UnknownIssuer #1130

Open CleanAir777 opened 4 months ago

CleanAir777 commented 4 months ago

Steps to Reproduce

I have installed rye successfully. I have done a "rye init ryetest" and "cd ryetest". All good so far.

I have than added the [[sources]] entry to ~/.rye/config.toml to point to the correct url for our corporate "mirror" of pypi.

When I then run "rye sync" I get error messages: scherm 20240531_042550

When I access the url directly, I can reach it without error. When doing a "pip install hatchling" it does download and install "hatchling"

Expected Result

The environment to successfully make contact with the local repository, and install the environment.

Actual Result

See above.

Version Info

afbeelding

Stacktrace

No response

bigbrozer commented 2 months ago

I guess your mirror has a self-signed certificate. Maybe try to add it to global trust store like /etc/ssl/certs/ca-certificates.crt.

If running Debian / Ubuntu:

  1. Copy the certificates in PEM format with the *.crt extension to /usr/local/share/ca-certificates. There should be one certificate per file, and not multiple certificates in a single file.
  2. Run sudo update-ca-certificates to merge the new certificates into the existing machine store at /etc/ssl/certs/ca-certificates.crt.
  3. Set environment variable export SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt.

Now, running rye (which use uv under the hood by default) should establish a verified TLS session to the host.

tacahiroy commented 2 months ago

@bigbrozer Your comment saved my life! On my Fedora 40 environment, this resolves the problem. (The path to the CA bundle file is different from Debian/Ubuntu.)

SSL_CERT_FILE=/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem rye sync
notenti commented 2 months ago

Anyone running into a similar issue during bootstrapping? There doesn't seem to be a way (either via rye's config, or uv's config) to specify an alternate source (company mirror, etc.) during the rye self install process. Here's what I've tried:

❯ SSL_CERT_FILE=<PATH_TO_CERT.PEM> UV_INDEX_URL=<URL_TO_MIRROR> PIP_INDEX_URL=<URL_TO_MIRROR> curl -sSf https://rye.astral.sh/get | bash
This script will automatically download and install rye (latest) for you.
######################################################################## 100.0%
Welcome to Rye!

This installer will install rye to /Users/<USER>/.rye
This path can be changed by exporting the RYE_HOME environment variable.

Details:
  Rye Version: 0.36.0
  Platform: macos (x86_64)

✔ Continue? · yes
✔ Select the preferred package installer · uv (fast, recommended)
✔ What should running `python` or `python3` do when you are not inside a Rye managed project? · Run the old default Python (provided by your OS, pyenv, etc.)
Installed binary to /Users/<USER>/.rye/shims/rye
Bootstrapping rye internals
Downloading cpython@3.12.3
Checking checksum
Unpacking
Downloaded cpython@3.12.3
error: Failed to download `pip==23.3.2`
  Caused by: Request failed after 3 retries
  Caused by: error sending request for url (https://files.pythonhosted.org/packages/15/aa/3f4c7bcee2057a76562a5b33ecbd199be08cdb4443a02e26bd2c3cf6fc39/pip-23.3.2-py3-none-any.whl.metadata)
  Caused by: client error (Connect)
  Caused by: invalid peer certificate: UnknownIssuer
error: Failed to download `tomli==2.0.1`
  Caused by: Request failed after 3 retries
  Caused by: error sending request for url (https://files.pythonhosted.org/packages/97/75/10a9ebee3fd790d20926a90a2547f0bf78f371b2f13aa822c759680ca7b9/tomli-2.0.1-py3-none-any.whl.metadata)
  Caused by: client error (Connect)
  Caused by: invalid peer certificate: UnknownIssuer
Updated self-python installation at /Users/<USER>/.rye/self

All done!

I kinda threw the kitchen sink at it with SSL_CERT_FILE, UV_INDEX_URL, and even PIP_INDEX_URL as a last-ditch effort. Note that even though package installation technically fails, rye does still install and is still available to use...but I have a feeling the broken bootstrap will come back to bite me eventually.

bigbrozer commented 2 months ago

Hello @notenti, just quickly looked to your comment but I think you are not passing environment variables to the Rye installer but to curl instead which does nothing... try with:

curl -sSf https://rye.astral.sh/get | SSL_CERT_FILE=<PATH_TO_CERT.PEM> UV_INDEX_URL=<URL_TO_MIRROR> PIP_INDEX_URL=<URL_TO_MIRROR> bash

Edit: You may need to export them if it does not work as the bash installer will call rye self install which may not "see" those variables...

notenti commented 2 months ago

Hello @notenti, just quickly looked to your comment but I think you are not passing environment variables to the Rye installer but to curl instead which does nothing... try with:

curl -sSf https://rye.astral.sh/get | SSL_CERT_FILE=<PATH_TO_CERT.PEM> UV_INDEX_URL=<URL_TO_MIRROR> PIP_INDEX_URL=<URL_TO_MIRROR> bash

Oh my goodness, you're right. Let me give it a whirl.

notenti commented 2 months ago

@bigbrozer worked like a charm. And to think I get paid for this...

For posterity, what I ended up doing is this:

curl -sSf https://rye.astral.sh/get | SSL_CERT_FILE=<PATH_TO_CERT.PEM> UV_INDEX_URL=<URL_TO_MIRROR> bash

I left off PIP_INDEX_URL as it's (rightfully) not needed. It looks like rye was able to "see" the env vars, but if anyone runs into a similar issue in the future, it may be worth just exporting the env vars instead of prepending them to the command.

Thanks for the help!

bigbrozer commented 2 months ago

@notenti that's cool ! Glad to help 😄

bigbrozer commented 2 months ago

@CleanAir777 are you able to test again using SSL_CERT_FILE ?

CleanAir777 commented 2 months ago

I will try to test this, yes.

Sill, I wonder if it will work because of two things:

  1. I am able to reach github, but astral.sh is not on the company whitelist (and I am not able to get it on the whitelist)
  2. The proxy I need to use works fine with poetry and pip, but not with rye (though I think the message comes from uv, not rye).
bigbrozer commented 2 months ago
1. I am able to reach github, but astral.sh is not on the company whitelist (and I am not able to get it on the whitelist)

Looks like we are working in the same company LOL. That's not an issue, the guys behind the project are doing great things. You can use the manual installation to download the Rye binary from Github releases, uncompress the archive and run rye self install (adapt options if needed).

2. The proxy I need to use works fine with poetry and pip, but not with rye (though I think the message comes from uv, not rye).

We are using a proxy with SSL inspection here. Proxy works well with rye / uv. The only thing was that SSL_CERT_FILE that contain the self-signed cert of the SSL inspection.

CleanAir777 commented 2 months ago

First, I went to the repository proxy that my company uses. Then I checked the certificate and downloaded the .PEM for the whole chain. I set the env var SSL_CERT_FILE via 'set -x' (fish). Then when trying 'uv pip install hatchling' I get the same error. Just running 'pip install hatchling' just works.

flurin137 commented 2 months ago

In case someone is having this same issue on Windows / Powershell:

$env:SSL_CERT_FILE="C:\path\to\cert.cer"
rye sync

might help

bigbrozer commented 1 month ago

First, I went to the repository proxy that my company uses. Then I checked the certificate and downloaded the .PEM for the whole chain. I set the env var SSL_CERT_FILE via 'set -x' (fish). Then when trying 'uv pip install hatchling' I get the same error. Just running 'pip install hatchling' just works.

I guess, downloading the PEM chain will contain only that certificate and not the whole bunch of public CAs that are used across the Internet. That would explain why pip is working (because you do not specify it so it uses the system one). You should really have a full chain that includes your CAs AND the public ones (Mozilla, etc...).

appleparan commented 4 weeks ago

For me, this issue was caused by bug of uv, and I found a workaround.

Additionally, from uv version 0.3.5, it supports the --trusted-host option, which allows you to bypass certificate verification. Someday, if the bundled uv gets updated, we'll be able to use it.