Add support for "manual" package indexes

[NoahMDMi]

Overview

I have a manual repository running in IIS on Windows server. Installing packages from the repository works fine with pip when specified as an extra-index-url, but when using uv pip, it seems to be unable to determine which versions are available, and thus does not fails to resolve.

Note that this is a manual repository not a simple repository, which has API endpoints. (Though in my limited experimentation with devpi, uv also was unable to resolve dependencies)

Is there any way to fix this? I love ruff and would be interested in using uv as well :smile:

Thanks!

Using pip

>> pip install extabpylish --verbose --dry-run                                                         
Using pip 24.1 from C:\Users\████████\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip (python 3.11)
Looking in indexes: https://pypi.org/simple, http://[manual-repository]/packages
Collecting extabpylish
  Using cached http://[manual-repository]/packages/extabpylish/extabpylish-0.1.0-py3-none-any.whl (8.5 kB)
Collecting openpyxl<4.0.0,>=3.1.2 (from extabpylish)
  Obtaining dependency information for openpyxl<4.0.0,>=3.1.2 from https://files.pythonhosted.org/packages/c0/da/977ded879c29cbd04de313843e76868e6e13408a94ed6b987245dc7c8506/openpyxl-3.1.5-py2.py3-none-any.whl.metadata
  Using cached openpyxl-3.1.5-py2.py3-none-any.whl.metadata (2.5 kB)
Collecting ordered-set<5.0.0,>=4.1.0 (from extabpylish)
  Obtaining dependency information for ordered-set<5.0.0,>=4.1.0 from https://files.pythonhosted.org/packages/33/55/af02708f230eb77084a299d7b08175cff006dea4f2721074b92cdb0296c0/ordered_set-4.1.0-py3-none-any.whl.metadata
  Using cached ordered_set-4.1.0-py3-none-any.whl.metadata (5.3 kB)
Collecting et-xmlfile (from openpyxl<4.0.0,>=3.1.2->extabpylish)
  Obtaining dependency information for et-xmlfile from https://files.pythonhosted.org/packages/96/c2/3dd434b0108730014f1b96fd286040dc3bcb70066346f7e01ec2ac95865f/et_xmlfile-1.1.0-py3-none-any.whl.metadata
  Using cached et_xmlfile-1.1.0-py3-none-any.whl.metadata (1.8 kB)
Using cached openpyxl-3.1.5-py2.py3-none-any.whl (250 kB)
Using cached ordered_set-4.1.0-py3-none-any.whl (7.6 kB)
Using cached et_xmlfile-1.1.0-py3-none-any.whl (4.7 kB)
Would install ExTabPylish-0.1.0 et-xmlfile-1.1.0 openpyxl-3.1.5 ordered-set-4.1.0

Using uv pip

>> uv pip install extabpylish --verbose --dry-run
DEBUG uv 0.2.23
DEBUG Searching for Python interpreter in system path or `py` launcher
TRACE Ignoring stale interpreter markers for: .venv\Scripts\python.exe
TRACE Querying interpreter executable at C:\Users\███████████████████████████████████████████\.venv\Scripts\python.exe
DEBUG Found cpython 3.11.9 at `C:\Users\███████████████████████████████████████████\.venv\Scripts\python.exe` (virtual environment)
DEBUG Using Python 3.11.9 environment at .venv\Scripts\python.exe
TRACE Checking lock for `.venv`
DEBUG Acquired lock for `.venv`
DEBUG At least one requirement is not satisfied: extabpylish
DEBUG Using request timeout of 30s
DEBUG Solving with installed Python version: 3.11.9
DEBUG Adding direct dependency: extabpylish*
TRACE Fetching metadata for extabpylish from http://[manual-repository]/packages/extabpylish/
TRACE cached request http://[manual-repository]/packages/extabpylish/ is storable because its response has a heuristically cacheable status code 200
TRACE cached request http://[manual-repository]/packages/extabpylish/ has a cached response that does not allow staleness
TRACE request http://[manual-repository]/packages/extabpylish/ does not have a fresh cache because its age is 263079 seconds, it is greater than the freshness lifetime of 0 seconds and stale cached responses are not allowed
DEBUG Found stale response for: http://[manual-repository]/packages/extabpylish/
DEBUG Sending revalidation request for: http://[manual-repository]/packages/extabpylish/
TRACE Handling request for http://[manual-repository]/packages/extabpylish/
TRACE Request for http://[manual-repository]/packages/extabpylish/ is unauthenticated, checking cache
TRACE No credentials in cache for URL http://[manual-repository]/packages/extabpylish/
TRACE Attempting unauthenticated request for http://[manual-repository]/packages/extabpylish/
TRACE is modified because status is 200 and not 304
DEBUG Found modified response for: http://[manual-repository]/packages/extabpylish/
TRACE cached request http://[manual-repository]/packages/extabpylish/ is storable because its response has a heuristically cacheable status code 200
TRACE Received package metadata for: extabpylish
TRACE selecting candidate for package extabpylish with range Range { segments: [(Unbounded, Unbounded)] } with 0 remote versions
DEBUG Searching for a compatible version of extabpylish (*)
TRACE exhausted all candidates for package PackageName("extabpylish") with range Range { segments: [(Unbounded, Unbounded)] } after 0 steps
TRACE selecting candidate for package extabpylish with range Range { segments: [(Unbounded, Unbounded)] } with 0 remote versions
TRACE exhausted all candidates for package PackageName("extabpylish") with range Range { segments: [(Unbounded, Unbounded)] } after 0 steps
DEBUG No compatible version found for: extabpylish
  × No solution found when resolving dependencies:
  ╰─▶ Because there are no versions of extabpylish and you require extabpylish, we can conclude that the requirements
      are unsatisfiable.

uv details

Platform: Windows 10 Version: uv 0.2.23 (4bc36c0cb 2024-07-08)

[ffolkes1911]

We are using devpi server and having problems as well.
First problem was that extra-index-url was failing to find package (whereas pip was successful), however using find-links worked, so we did not pay much attention.
Yesterday we tried to bump minor version 0.0.26 -> 0.1.0 (tried 1.0.0 as well) and as long as that new package (0.1.0) is in devpi we cannot install any version of our package.
Problems persist with older (0.1.39) and most recent (0.2.22) version of uv.

[charliermarsh]

Can you say more about your setup? Which index is the index-url and which is the extra-index-url? Does the package exist on both? Have you tried using the index-strategy setting?

[NoahMDMi]

In my case, we have packages that are only available on the manual repository. We specify the manual repository as the extra-index-url. The index-url is the default, PyPI.

I believe pip checks the extra-index-url after it is unable to find the package on the index-url, which is desired, since our requirements.txt refers to packages from PyPI, and from the manual repository (no overlap).

I'm not sure the index-strategy setting applies here, since uv pip is checking the extra-index-url, it's just not finding the packages/versions (where pip finds them without issue)

Let me know if there's any more information I can provide! 😄

[ffolkes1911]

We have our package only in our internal (http) devpi server as well, which is the "extra" index (provided via find-links, because extra-index-url did not work). Have not tried index-strategy, but we tried installing specifically only that package via CLI to no success, and I assume that index-strategy does not influence this scenario.

[ziima]

I can confirm the same problem. uv for some reason doesn't work with non-simple devpi indexes.

Here's what I tried:

pip install --index-url 'https://example.cz/me/test' 'coverage' --force-reinstall  # Works

uv pip install --index-url 'https://example.cz/me/test' 'coverage' --force-reinstall  # Doesn't work
uv pip install --extra-index-url 'https://example.cz/me/test' 'coverage' --force-reinstall  # Doesn't work
uv pip install --index-url 'https://example.cz/me/test/+simple/' 'coverage' --force-reinstall  # Works
uv pip install --extra-index-url 'https://example.cz/me/test/+simple/' 'coverage' --force-reinstall  # Works
uv pip install --find-links 'https://example.cz/me/test' 'coverage' --force-reinstall  # Works

It's not really easy to find out, simple index needs to be used (see also #706).

[NoahMDMi]

Any update on this? Updated to the latest version of uv (0.3.0) and I am still seeing this issue :(

[zanieb]

We don't have support for non-simple indexes. It seems reasonable to add support, but it's not high priority.