cachix / devenv

Fast, Declarative, Reproducible, and Composable Developer Environments
https://devenv.sh
Apache License 2.0
3.74k stars 276 forks source link

Add support for the `rye` python package manager #1318

Open anoadragon453 opened 3 days ago

anoadragon453 commented 3 days ago

rye is a comprehensive project and package manager for Python. It makes use of uv, a rust rewrite of pip, and supports large workspaces and monorepos.

rye manages everything; the python version, installing dependencies, linting your code with ruff, and running unit tests with pytest.

I would like to see integration of rye with devenv to a similar level as poetry and venv.


We've started using rye at my company. I took an initial stab at integrating it into a devenv env, but some difficulty arose due to rye downloading pre-built binaries of python, uv and ruff from the internet. The binaries are dynamically linked against the system'd ld, and thus are incompatible with NixOS.

Click to see my current devenv module ```nix { # Configure packages to install. # Search for package names at https://search.nixos.org/packages?channel=unstable packages = with pkgs; [ # Manages python and uv versions, and builds the venv. rye # Required by uv. libz ]; # A dirty script to patch any binaries `rye` may have decided to download. # Best to be run after `rye sync`. scripts.patch-binaries.exec = '' # Paths to patchelf and glibc PATCHELF="${pkgs.patchelf}/bin/patchelf" GLIBC_INTERPRETER="${pkgs.glibc}/lib/ld-linux-x86-64.so.2" # Iterate over each binary that `rye` has downloaded. for FILE in "$HOME/.rye/self/bin"/*; do # Check if the file is an ELF executable if file "$FILE" | grep -q "ELF"; then # Run patchelf on the ELF executable "$PATCHELF" --set-interpreter "$GLIBC_INTERPRETER" "$FILE" fi done ''; enterShell = '' # Allow `uv` (and other binaries) to find the libraries installed # in the devenv profile. export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$DEVENV_ROOT/.devenv/profile/lib # Only do the following if we're on NixOS. if command -v nixos-version >/dev/null 2>&1; then # Ensure that the `uv` and `python` binaries that rye downloads can # be run on NixOS. # Download and patch the `python` binary at the version required # by the project. rye toolchain fetch ${pkgs.patchelf}/bin/patchelf \ --set-interpreter ${pkgs.glibc}/lib/ld-linux-x86-64.so.2 \ $HOME/.rye/py/cpython@*/bin/python3 # Download and patch the `uv` binary at the version required # by the project. # # We expect this command to fail, as `rye` will download `uv` # (which we want) and then attempt to execute it. The latter # will fail, as it has not yet been patched. # # The failure looks quite scary, so let's pipe all output to # /dev/null to hide it. Hopefully this command never needs to be # debugged :) rye tools install uv &>/dev/null || echo -n "" ${pkgs.patchelf}/bin/patchelf \ --set-interpreter ${pkgs.glibc}/lib/ld-linux-x86-64.so.2 \ $HOME/.rye/uv/*/uv fi ''; } ```

As you can see, there's a lot going into just getting the binaries patched for NixOS. This works, and entering the development environment is pretty fast. But rye sync must still be run manually, and patch-binaries must be run whenever rye attempts to download a new binary.

I'm presenting my code here in an effort to get more eyes on it, and eventually have a rye option in devenv that's as simple as languages.python.poetry.enable. Where the rye-managed python environment will be built for you, and rebuilt when the dependencies are changed.

domenkozar commented 3 days ago

Is there a mode in Rye where it would let devenv manage dependencies?

domenkozar commented 3 days ago

Maybe @mitsuhiko can help here.

anoadragon453 commented 3 days ago

@domenkozar What do you mean exactly? There's some documentation here: https://rye.astral.sh/guide/sync/

ryes "lockfile" is just a standard requirements.txt, but named requirements.lock. So one can just create a venv and then install all the locked dependencies with pip install -r requirements.lock. Note that there it also supports a requirements-dev.lock.

rye can also work with an existing venv - it expects it to live at .venv in the project root. If users run rye sync, it will install the dependencies from the lockfiles into .venv if needed. It will also attempt to download appropriate python, uv and ruff binaries if not already present.

These binaries are installed to ~/.rye in the user's home directory by default.

domenkozar commented 2 days ago

Shouldn't we then integrate it with out existing venv so none of the patching is needed?