Support pre-releases in `pypi-dependencies`

[ksquarekumar]

Checks

• [X] I have checked that this issue has not already been reported.

• [X] I have confirmed this bug exists on the latest version of pixi, using pixi --version.

Reproducible example

pixi info

❯ pixi info
      Pixi version: 0.20.1
          Platform: linux-64
  Virtual packages: __unix=0=0
                  : __linux=6.7.6=0
                  : __glibc=2.38=0
                  : __cuda=12.4=0
                  : __archspec=1=skylake
         Cache dir: /home/ksquare/.cache/rattler/cache
      Auth storage: /home/ksquare/.rattler/credentials.json

Project
------------
           Version: 0.0.1
     Manifest file: /shared/git/redacted/experiments/tensorrt_llm/pixi.toml
  Config locations: /shared/git/redacted/experiments/tensorrt_llm/.pixi/config.toml
      Last updated: 28-04-2024 00:55:41

Environments
------------
       Environment: default
          Features: default
          Channels: nvidia, conda-forge, pytorch
  Dependency count: 11
      Dependencies: python, datasets, evaluate, sentencepiece, zlib, libpq, openssl, compilers, make, cmake, rust
 PyPI Dependencies: tensorrt_llm, tensorrt, rouge_score
  Target platforms: linux-64

       Environment: dev
          Features: dev, default
       Solve group: default
          Channels: nvidia, conda-forge, pytorch
  Dependency count: 24
      Dependencies: pygls, nox, ruff, mypy, mypy_extensions, matplotlib, jupyterlab, jupyter_client, ipywidgets, notebook, ipykernel, debugpy, ipython, python, datasets, evaluate, sentencepiece, zlib, libpq, openssl, compilers, make, cmake, rust
 PyPI Dependencies: huggingface_hub, pip, black, blacken-docs, prompt_toolkit, platformdirs, virtualenv, packaging, pyright, wheel, commitizen, detect-secrets, types-beautifulsoup4, types-requests, types-PyYAML, types-decorator, types-six, types-tabulate, types-orjson, types-ujson, types-protobuf, tensorrt_llm, tensorrt, rouge_score
  Target platforms: linux-64

pixi shell -e dev

error: unexpected argument '--prerelease=allow' found

Usage: pixi install [OPTIONS]

For more information, try '--help'.
  ⠒ dev:linux-64         [00:00:04] resolving conda                                                                                                                                                                                                                            x failed to solve the pypi requirements of 'default' 'linux-64'
  |-> failed to resolve pypi dependencies
  `-> Because tensorrt-llm==0.9.0 depends on tensorrt==9.3.0.post12.dev1 and there is no version of tensorrt==9.3.0.post12.dev1, we can conclude that tensorrt-llm==0.9.0 cannot be used.
      And because you require tensorrt-llm==0.9.0, we can conclude that the requirements are unsatisfiable.

      hint: tensorrt was requested with a pre-release marker (e.g., tensorrt==9.3.0.post12.dev1), but pre-releases weren't enabled (try: `--prerelease=allow`)

pixi.toml

#:schema: https://github.com/prefix-dev/pixi/blob/main/schema/schema.json
[project]
name = 'tensorrt-llm'
version = '0.0.1'
description = 'Environment for running TensorRT LLM'
readme = 'README.md'
license = "NOASSERTION"
authors = ["<REDACTED>"]
homepage = "<REDACTED>"
platforms = ["linux-64"]
channels = ["nvidia", "conda-forge", {channel = "pytorch", priority = -1}]

[dependencies]
python = "==3.10.0"
datasets = "*"
evaluate = "*"
sentencepiece = "*"

[activation]
scripts = ["../../scripts/activation-linux64-cuda.sh"]

[pypi-dependencies]
tensorrt_llm = "==0.9.0"
tensorrt = "==9.3.0.*"
rouge_score = "*"

[host-dependencies]
zlib = ">=1.0.0"
libpq = ">=16.0"
openssl = ">=3.2.0"
compilers = "*"
make = "*"
cmake = "*"
rust = "*"

[system-requirements]
linux = "6.0"
cuda = "12.1"

[feature.dev.dependencies]
pygls = ">=1.3.1"
nox = ">=2024.4.0"
ruff = ">=0.4.1"
mypy = ">=1.9.0"
mypy_extensions = ">=1.0.0"
matplotlib = ">=3.8.4"
jupyterlab = ">=4.1.6"
jupyter_client = ">=8.6.1"
ipywidgets = ">=8.1.2"
notebook = ">=7.1.2"
ipykernel = ">=6.29.0"
debugpy = ">=1.6.5"
ipython = ">=8.0.0"

[feature.dev.pypi-dependencies]
huggingface_hub = { version = ">=0.22.0", extras = ["cli", "hf_transfer"] }
pip = ">=24.0.0"
black = ">=24.0.0"
blacken-docs = ">=1.16.0"
prompt_toolkit = ""
platformdirs = "*"
virtualenv = "*"
packaging = "*"
pyright = "*"
wheel = "*"
commitizen = "*"
detect-secrets = "*"
types-beautifulsoup4 = "*"
types-requests = "*"
types-PyYAML = "*"
types-decorator = "*"
types-six = "*"
types-tabulate = "*"
types-orjson = "*"
types-ujson = "*"
types-protobuf = "*"

[environments]
dev = { features = ["dev"], solve-group= "default" }

Issue description

Environments cannot be resolved when underlying package requirements constraints are marked pre-release, in this example I am assuming I simply can't install tensorrt_llm package in the environment, because a flag that the underlying tool uv could have accepted to solve the constraints are not possible to pass with pixi install ?

Expected behavior

Either have full parity with the interfaces of the underlying implementations / tools (hard to do when both are new and rapidly moving targets), or allow the user to have some flexibility to tackle such situations by themselves because the tooling isn't there yet, i should be able to pass flags over to uv to get out of this situation in this case.

[ruben-arts]

He @ksquarekumar

We're not running uvs binary but it is baked into pixi through their rust crates so it is not as simple as pushing flags and options to the cli as we don't have the uv cli available. Unfortunately for us uv has backed in some of their tips into the errors that we can encounter aswell, so they are not helpful for the users of pixi in a lot of the cases.

For the prereleases I'll have to get back to you!

[RaphaelMelanconAtBentley]

Any plan to add this at some point?

[ruben-arts]

If you depend on a pre-release version this already works. e.g.

[project]
name = "pre"
channels = ["conda-forge"]
platforms = ["linux-64"]

[dependencies]
python = ">=3.12.4,<4"

[pypi-dependencies]
django = "==5.1rc1"

[RaphaelMelanconAtBentley]

What about transitive dependencies? azure-cli (not pre-release) depends on pre-release versions of other packages.

[tdejager]

Sure, we could expose https://docs.astral.sh/uv/reference/settings/#prerelease if that would help.

[PanTheDev]

@tdejager I think this is necessary, otherwise any packages that depend on pre-releases and are not on conda-forge can't be installed. Although it's a bad practice for a release version to depend on pre-releases, I guess it can't always be avoided...