prefix-dev / pixi

Package management made easy
https://pixi.sh
BSD 3-Clause "New" or "Revised" License
2.97k stars 164 forks source link

Allow for installation of yanked PyPI packages #2077

Open MG-MW opened 4 days ago

MG-MW commented 4 days ago

Checks

The pixi version is 0.29.0.

Reproducible example

$ pixi init && pixi add "python==3.11" && pixi add "setuptools-scm==6.2.0" --pypi
✔ Created /tmp/test_pixi_yanked/pixi.toml
✔ Added python==3.11
  × failed to solve the pypi requirements of 'default' 'linux-64'
  ├─▶ failed to resolve pypi dependencies
  ╰─▶ Because setuptools-scm==6.2.0 was yanked (reason: regression due to a discovered setuptools bug) and you require setuptools-
      scm==6.2.0, we can conclude that your requirements are unsatisfiable.

Issue description

This is, what the official python packaging guide says about this topic: https://packaging.python.org/en/latest/specifications/simple-repository-api/#installers

What this means is left up to the specific installer, to decide how to best fit into the overall usage of their installer. However, there are two suggested approaches to take:

  1. Yanked files are always ignored, unless they are the only file that matches a version specifier that “pins” to an exact version using either == (without any modifiers that make it a range, such as .*) or ===. Matching this version specifier should otherwise be done as per the version specifiers specification for things like local versions, zero padding, etc.
  2. Yanked files are always ignored, unless they are the only file that matches what a lock file (such as Pipfile.lock or poetry.lock) specifies to be installed. In this case, a yanked file SHOULD not be used when creating or updating a lock file from some input file or command.

pixi currently takes the second recommended approach, but pip and uv follow the first one.

Usual pip behaviour with a specified version is to install the yanked package but give a warning. There are various reasons to yank a package and sometimes they are not important to your case. If you run the equivilant commands with pip, you get:

$ python -m venv .venv2 && ./.venv2/bin/python -m pip install "setuptools-scm==6.2.0"
Collecting setuptools-scm==6.2.0
  Downloading setuptools_scm-6.2.0-py3-none-any.whl.metadata (22 kB)
Collecting setuptools>=45 (from setuptools-scm==6.2.0)
  Using cached setuptools-75.1.0-py3-none-any.whl.metadata (6.9 kB)
Collecting tomli>=1.0 (from setuptools-scm==6.2.0)
  Using cached tomli-2.0.1-py3-none-any.whl.metadata (8.9 kB)
WARNING: The candidate selected for download or install is a yanked version: 'setuptools-scm' candidate (version 6.2.0 at https://files.pythonhosted.org/packages/6d/85/4436669ef4d143cdd77426cd72f62e456cd51cfe8b55fd246503d3cbc5d9/setuptools_scm-6.2.0-py3-none-any.whl (from https://pypi.org/simple/setuptools-scm/) (requires-python:>=3.6))
Reason for being yanked: regression due to a discovered setuptools bug
Downloading setuptools_scm-6.2.0-py3-none-any.whl (32 kB)
Using cached setuptools-75.1.0-py3-none-any.whl (1.2 MB)
Using cached tomli-2.0.1-py3-none-any.whl (12 kB)
Installing collected packages: tomli, setuptools, setuptools-scm
Successfully installed setuptools-75.1.0 setuptools-scm-6.2.0 tomli-2.0.1

[notice] A new release of pip is available: 24.0 -> 24.2
[notice] To update, run: python -m pip install --upgrade pip

If you run the equivilant commands with uv, you get:

$ uv venv --python 3.11 && uv pip install "setuptools-scm==6.2.0"
Using Python 3.11.9
Creating virtualenv at: .venv
Activate with: source .venv/bin/activate
Resolved 3 packages in 1.43s
Prepared 1 package in 18ms
░░░░░░░░░░░░░░░░░░░░ [0/3] Installing wheels...                                                                                          warning: Failed to hardlink files; falling back to full copy. This may lead to degraded performance.
         If the cache and target directories are on different filesystems, hardlinking may not be supported.
         If this is intentional, set `export UV_LINK_MODE=copy` or use `--link-mode=copy` to suppress this warning.
Installed 3 packages in 25ms
 + setuptools==75.1.0
 + setuptools-scm==6.2.0
 + tomli==2.0.1
warning: `setuptools-scm==6.2.0` is yanked (reason: "regression due to a discovered setuptools bug")

Expected behavior

Similar to pip and uv install the yanked version, if the version is fully specified, but give a warning.

tdejager commented 3 days ago

Seems like a good idea, I just looked into how uv does it and we can do something similar for pixi. And thanks for reporting, this is useful!