search

astral-sh / uv

An extremely fast Python package and project manager, written in Rust.
https://docs.astral.sh/uv
Apache License 2.0
24.32k stars 704 forks source link

Private git dependency URLs with credentials do not resolve #1514

Closed meridionaljet closed 8 months ago

meridionaljet commented 8 months ago

Working with private git repos that require a username and token to resolve seems to fail in uv:

uv pip install '{packagename} @ git+https://{username}@github.com/{organization}/{packagename}.git'
Updating https://{username}/ (github.com/{organization}/{packagename}.git) 
error: Failed to download and build: {packagename} @ git+https://{username}@github.com/{organization}/{packagename}.git
  Caused by: Git operation failed
  Caused by: failed to fetch into: /home/user/.cache/uv/git-v0/db/20cbf189775078d1
  Caused by: failed to connect to the repository
  Caused by: failed to resolve address for {username}: Temporary failure in name resolution; class=Net (12)

pip will read ~/.git-credentials and find the token associated with username and resolve the URL correctly. pip will also accept the token directly using the form pip install '{packagename} @ git+https://{username}:{token}@github.com/{organization}/{packagename}.git', but uv complains about the URL format in this case, trying to interpret token as a port:

thread 'main' panicked at crates/cache-key/src/canonical_url.rs:113:38:
called `Result::unwrap()` on an `Err` value: InvalidPort
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
SnoopJ commented 8 months ago

I have seen a similar problem when attempting to install pkg @ git+ssh://git@github.com/… where github.com has a corresponding Host entry in ~/.ssh/config

Dimitrioglo commented 8 months ago

Did you try to use GITHUB_ACTOR and GITHUB_TOKEN for auth? For me works well

related info:

lexicalunit commented 8 months ago

I'm also having this issue with dependencies with URLs that begin like git+ssh://git, for example:

cat > reqs.txt <<EOF
elasticmock @ git+ssh://git@github.com/monte-carlo-data/elasticmock.git@1.11.0
EOF

uv pip install -r reqs.txt

 uv::requirements::from_source source=reqs.txt
    0.000328s DEBUG uv_interpreter::virtual_env Found a virtualenv through VIRTUAL_ENV at: /Users/me/source/montecarlo/monolith-django/venv
    0.000524s DEBUG uv_interpreter::interpreter Using cached markers for: /Users/me/source/montecarlo/monolith-django/venv/bin/python
    0.000532s DEBUG uv::commands::pip_install Using Python 3.8.17 environment at /Users/me/source/montecarlo/monolith-django/venv/bin/python
 uv_client::flat_index::from_entries
 uv_resolver::resolver::solve
      0.138836s   0ms DEBUG uv_resolver::resolver Solving with target Python version 3.8.17
   uv_resolver::resolver::choose_version package=root
   uv_resolver::resolver::get_dependencies package=root, version=0a0.dev0
        0.138896s   0ms DEBUG uv_resolver::resolver Adding direct dependency: elasticmock*
   uv_resolver::resolver::choose_version package=elasticmock
        0.138939s   0ms DEBUG uv_resolver::resolver Searching for a compatible version of elasticmock @ git+ssh://git@github.com/monte-carlo-data/elasticmock.git@1.11.0 (*)
 uv_resolver::resolver::process_request request=Metadata elasticmock @ git+ssh://git@github.com/monte-carlo-data/elasticmock.git@1.11.0
   uv_distribution::distribution_database::get_or_build_wheel_metadata dist=elasticmock @ git+ssh://git@github.com/monte-carlo-data/elasticmock.git@1.11.0
    0.261561s DEBUG uv_git::source Updating git source `Url { scheme: "ssh", cannot_be_a_base: false, username: "git", password: None, host: Some(Domain("github.com")), port: None, path: "/monte-carlo-data/elasticmock.git", query: None, fragment: None }`
    0.262142s DEBUG uv_git::git Attempting GitHub fast path for: https://api.github.com/repos/monte-carlo-data/elasticmock/commits/1.11.0
    0.299788s DEBUG uv_git::git skipping gc as there's only 0 pack files
    0.299888s DEBUG uv_git::git Performing a Git fetch for: ssh://git@github.com/monte-carlo-data/elasticmock.git
    0.300004s DEBUG uv_git::git initiating fetch of ["+refs/heads/1.11.0:refs/remotes/origin/1.11.0", "+refs/tags/1.11.0:refs/remotes/origin/tags/1.11.0"] from ssh://git@github.com/monte-carlo-data/elasticmock.git
    0.849429s DEBUG uv_git::git fetch failed: no authentication methods succeeded
error: Failed to download and build: elasticmock @ git+ssh://git@github.com/monte-carlo-data/elasticmock.git@1.11.0
  Caused by: Git operation failed
  Caused by: failed to fetch into: /Users/me/Library/Caches/uv/git-v0/db/83a8ffe9f36e5f2e
  Caused by: failed to authenticate when downloading repository

* attempted ssh-agent authentication, but no usernames succeeded: `git`
  Caused by: no authentication methods succeeded

Exit code is 2.

Compared to output from pip install:

pip install -v -r reqs.txt
Using pip 24.0 from /Users/me/source/montecarlo/monolith-django/venv/lib/python3.8/site-packages/pip (python 3.8)
Collecting elasticmock@ git+ssh://****@github.com/monte-carlo-data/elasticmock.git@1.11.0 (from -r reqs.txt (line 1))
  Cloning ssh://****@github.com/monte-carlo-data/elasticmock.git (to revision 1.11.0) to /private/var/folders/k6/l90cqg1x7jj20qf91bqhh4p40000gn/T/pip-install-cmkzk7fk/elasticmock_e2963d61c270497da34f03732656cb01
  Running command git version
  git version 2.39.0
  Running command git clone --filter=blob:none 'ssh://****@github.com/monte-carlo-data/elasticmock.git' /private/var/folders/k6/l90cqg1x7jj20qf91bqhh4p40000gn/T/pip-install-cmkzk7fk/elasticmock_e2963d61c270497da34f03732656cb01
  Cloning into '/private/var/folders/k6/l90cqg1x7jj20qf91bqhh4p40000gn/T/pip-install-cmkzk7fk/elasticmock_e2963d61c270497da34f03732656cb01'...
  Running command git show-ref 1.11.0
  ead2f1a441569c8df929e389449b355aa1bf8e1b refs/tags/1.11.0
  Running command git rev-parse HEAD
  ead2f1a441569c8df929e389449b355aa1bf8e1b
  Resolved ssh://****@github.com/monte-carlo-data/elasticmock.git to commit ead2f1a441569c8df929e389449b355aa1bf8e1b
  Running command git rev-parse HEAD
  ead2f1a441569c8df929e389449b355aa1bf8e1b
  Running command pip subprocess to install build dependencies
  Collecting setuptools>=40.8.0
    Using cached setuptools-69.1.0-py3-none-any.whl.metadata (6.1 kB)
  Using cached setuptools-69.1.0-py3-none-any.whl (819 kB)
  Installing collected packages: setuptools
  Successfully installed setuptools-69.1.0
  Installing build dependencies ... done
  Running command Getting requirements to build wheel
  running egg_info
  creating ElasticMock.egg-info
  writing ElasticMock.egg-info/PKG-INFO
  writing dependency_links to ElasticMock.egg-info/dependency_links.txt
  writing requirements to ElasticMock.egg-info/requires.txt
  writing top-level names to ElasticMock.egg-info/top_level.txt
  writing manifest file 'ElasticMock.egg-info/SOURCES.txt'
  reading manifest file 'ElasticMock.egg-info/SOURCES.txt'
  reading manifest template 'MANIFEST.in'
  adding license file 'LICENSE'
  writing manifest file 'ElasticMock.egg-info/SOURCES.txt'
  Getting requirements to build wheel ... done
  Running command pip subprocess to install backend dependencies
  Collecting wheel
    Using cached wheel-0.42.0-py3-none-any.whl.metadata (2.2 kB)
  Using cached wheel-0.42.0-py3-none-any.whl (65 kB)
  Installing collected packages: wheel
  Successfully installed wheel-0.42.0
  Installing backend dependencies ... done
  Running command Preparing metadata (pyproject.toml)
  running dist_info
  creating /private/var/folders/k6/l90cqg1x7jj20qf91bqhh4p40000gn/T/pip-modern-metadata-a9ptxrjl/ElasticMock.egg-info
  writing /private/var/folders/k6/l90cqg1x7jj20qf91bqhh4p40000gn/T/pip-modern-metadata-a9ptxrjl/ElasticMock.egg-info/PKG-INFO
  writing dependency_links to /private/var/folders/k6/l90cqg1x7jj20qf91bqhh4p40000gn/T/pip-modern-metadata-a9ptxrjl/ElasticMock.egg-info/dependency_links.txt
  writing requirements to /private/var/folders/k6/l90cqg1x7jj20qf91bqhh4p40000gn/T/pip-modern-metadata-a9ptxrjl/ElasticMock.egg-info/requires.txt
  writing top-level names to /private/var/folders/k6/l90cqg1x7jj20qf91bqhh4p40000gn/T/pip-modern-metadata-a9ptxrjl/ElasticMock.egg-info/top_level.txt
  writing manifest file '/private/var/folders/k6/l90cqg1x7jj20qf91bqhh4p40000gn/T/pip-modern-metadata-a9ptxrjl/ElasticMock.egg-info/SOURCES.txt'
  reading manifest file '/private/var/folders/k6/l90cqg1x7jj20qf91bqhh4p40000gn/T/pip-modern-metadata-a9ptxrjl/ElasticMock.egg-info/SOURCES.txt'
  reading manifest template 'MANIFEST.in'
  adding license file 'LICENSE'
  writing manifest file '/private/var/folders/k6/l90cqg1x7jj20qf91bqhh4p40000gn/T/pip-modern-metadata-a9ptxrjl/ElasticMock.egg-info/SOURCES.txt'
  creating '/private/var/folders/k6/l90cqg1x7jj20qf91bqhh4p40000gn/T/pip-modern-metadata-a9ptxrjl/ElasticMock-1.10.0.dist-info'
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: elasticsearch in ./venv/lib/python3.8/site-packages (from elasticmock@ git+ssh://git@github.com/monte-carlo-data/elasticmock.git@1.11.0->-r reqs.txt (line 1)) (8.12.0)
Requirement already satisfied: python-dateutil in ./venv/lib/python3.8/site-packages (from elasticmock@ git+ssh://git@github.com/monte-carlo-data/elasticmock.git@1.11.0->-r reqs.txt (line 1)) (2.8.2)
Requirement already satisfied: elastic-transport<9,>=8 in ./venv/lib/python3.8/site-packages (from elasticsearch->elasticmock@ git+ssh://git@github.com/monte-carlo-data/elasticmock.git@1.11.0->-r reqs.txt (line 1)) (8.12.0)
Requirement already satisfied: six>=1.5 in ./venv/lib/python3.8/site-packages (from python-dateutil->elasticmock@ git+ssh://git@github.com/monte-carlo-data/elasticmock.git@1.11.0->-r reqs.txt (line 1)) (1.16.0)
Requirement already satisfied: urllib3<3,>=1.26.2 in ./venv/lib/python3.8/site-packages (from elastic-transport<9,>=8->elasticsearch->elasticmock@ git+ssh://git@github.com/monte-carlo-data/elasticmock.git@1.11.0->-r reqs.txt (line 1)) (2.2.0)
Requirement already satisfied: certifi in ./venv/lib/python3.8/site-packages (from elastic-transport<9,>=8->elasticsearch->elasticmock@ git+ssh://git@github.com/monte-carlo-data/elasticmock.git@1.11.0->-r reqs.txt (line 1)) (2024.2.2)

Note that I do not have a ~/.ssh/config file. Nor do I have a ~/.git-credentials file.

Edit: I actually was able to resolve this by adding a properly configured (not just empty) ~/.ssh/config file. Not sure why the pip install worked without it. 🤷🏻‍♀️

lexicalunit commented 8 months ago

Hrm. Actually I can't get uv pip compile to work, even after adding a ~/.ssh/config file (which allowed pip install to work).

➜ uv pip compile reqs.in
Updating ssh://git@github.com/monte-carlo-data/elasticmock.git (1.11.0)                                                                                                                 error: Failed to download and build: elasticmock @ git+ssh://git@github.com/monte-carlo-data/elasticmock.git@1.11.0
  Caused by: Git operation failed
  Caused by: failed to fetch into: /Users/me/Library/Caches/uv/git-v0/db/83a8ffe9f36e5f2e
  Caused by: failed to authenticate when downloading repository

* attempted ssh-agent authentication, but no usernames succeeded: `git`
  Caused by: no authentication methods succeeded

Exit code 2.

TudorAndrei-Pythia commented 8 months ago

I think this issue is related to #1452

albertotb commented 8 months ago

I'm encountering the same issue. I can

But using uv pip install "doraemon @ git+ssh://git@github.com/Komorebi-AI/doraemon.git@1.0" throws the error:

error: Failed to download and build: doraemon @ git+ssh://git@github.com/Komorebi-AI/doraemon.git@1.0
  Caused by: Git operation failed
  Caused by: failed to fetch into: /home/users/atorres/.cache/uv/git-v0/db/85abc53470d01254
  Caused by: failed to authenticate when downloading repository

* attempted ssh-agent authentication, but no usernames succeeded: `git`
  Caused by: no authentication methods succeeded